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ABSTRACT 


Combat arms units (both Marine and Army) often do not have enough people, 
time and resources to properly train collective tasks at the squad level. Resources are 
often retained by higher headquarters due to tight deployment schedules, land 
restrictions, logistics constraints and a myriad of other reasons. Due to the current 
operational demands of combat arms brigades and regiments, the reality of limited 
resources is often a contributing factor in poor performance at the squad level. Leaders at 
all levels will need to look for innovative ways to sustain training levels at the small unit 
level. The scope of this study examined the collective and leader tasks that are required 
for successful execution of infantry squad missions (using the Army Training and 
Evaluation Plan - ARTEP 7-8 Drill), and how those tasks could be trained with the use of 
commercial off-the-shelf multiplayer gaming software. Using a group of Second 
Eieutenants enrolled at the U.S. Army Infantry Officer Basic Course, we conducted a 
training event using a commercial game as an additional training event to prepare them 
for their squad maneuver live-fire event. We found that the squads that conducted the 
additional training performed better than the squads that did not receive the games¬ 
training, and the level of user acceptance by our test squads was high. Overall, we were 
able to demonstrate that the potential of using commercial games can yield positive 
results, but the training event needs to be resourced properly (with training officers and 
NCOs present), structured (to include an AAR process), and the platform needs to be 
accepted as a training venue by the users in order to gain any value from the event. 
Euture work needs to address games-based training over the length of an entire training 
cycle, the type of platform best suited for training (PC, console or hand-held devices), 
and what additional skills that this type of training can be used to train. 
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I. INTRODUCTION 

A. PROBLEM STATEMENT 

The infantry community today lacks an accessible simulation tool for collective 
training events. These events prepare cohesive teams and units to accomplish the mission 
by integrating individual and crew tasks at the squad level. This tool must be cost 
effective, robust enough to create virtual environments that replicate the asymmetrical 
battlefield we face today, and have simple hardware requirements that do not tie units to 
fixed simulation centers, but can be run at any installation or forward operating base 
computer laboratory. 

B. HYPOTHESIS 

Multiplayer commercial “off the shelf’ first person shooter games can be 
effective for use by infantry squads as a low resource, high impact small unit training tool 
for the conduct of collective training. 

C. DEFINITION OF TERMS 

• Collective Training - training on collective tasks; collective tasks are 
those which require the to leader integrate lower-level individual and crew tasks 

• First Person Shooter - a computer or video game where the player's on¬ 
screen view of the game world simulates that of the character; genre of game with 
a first-person view, almost always centered around the act of aiming and shooting 
(Wikipedia, 2005) 

• Multiplayer - mode of play for computer and video games in which 
multiple people can play the same game at the same time (Wikipedia, 2005) 

D. MOTIVATION FOR RESEARCH 

I. Perspective on Infantry Training 

In 480 B.C., a force of some 300 infantrymen from Sparta held off the might of 
the Persian army, under the rule of King Xerxes, for over seven days at the pass at 
Thermopylae. The Spartans, known as fierce warriors in their own right, doubled the 
training of their forces and their Thespian allies to prepare for the closing Persian assault. 
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The Spartans prepared for the onslaught through the use of serious games (tough, realistic 
live and simulated collective training events) in order to prepare their warriors. 
Conducting exhausting picket and shield drills on the training field, practicing 
swordsmanship against simulated combatants (wooden dummies) and sparring partners, 
and honing hand to hand combat skills against each other, the Spartan force prepared for 
a fight to defend the very foundations of liberty and justice. Ultimately, the force of King 
Xerxes’ might proved too much for the Spartans at Thermopylae, and the Spartans fell 
after a week of bloodied combat. But before the Spartans fell, they killed untold 
thousands of Persian forces between the rocky mountain passes and sea below. Fighting 
to the last man, the Spartan Infantrymen, by their example, rallied the people of ancient 
Greece against the Persian invaders, and ultimately led to the preservation of the seeds of 
Western democracy and the freedom of man (Pressfield, 1999). Though a doomed 
mission from the onset, the Spartan’s skill and discipline, forged by serious games 
allowed them to accomplish what no other force in the ancient world had ever done - 
prove the Persian army was not invulnerable, and crush the momentum of King Xerxes’ 
conquest of the Greece. 

Each era of warfare brings unique and demanding challenges for our infantrymen. 
Although conditions change, tough and realistic training remains as constant a principle 
for development of the infantryman. As our nation conducts a new asymmetric fight in 
Southwest Asia, critical infantry collective combat training (once reserved for combat 
arms alone) is now extending to combat support and service support Soldiers and 
Marines. Demands on resources, even more limited due to obligations for current combat 
operations, place trainers in a difficult position balancing people, time, and training 
requirements. Because training resources are scarce, leaders at all levels are looking for 
innovative ways to conduct small unit collective training events. Due to Herculean 
efforts of the Non-Commissioned Officers Corps, individual Soldier and Marine skills 
continue to be maintained at the highest levels. Additionally, battalions and brigades 
continue to sustain a strong level of proficiency through use of virtual and constructive 
simulations such as the Brigade/Battalion Battle Simulation (BBS), Corps Battle 
Simulation (CBS) and other constructive simulation tools. The training gap that exists 
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rests between the skilled individual Soldier or Marine and his higher headquarters. 
Finding a cost effective way to overcome this gap to train small units on combat 
operations is the challenge of Marine and Army leadership at all levels. 

2. Bridging the Gap 

This study aims to demonstrate that multiplayer commercial off-the-shelf (COTS) 
gaming software, in conjunction with existing unit training plans, can bridge the training 
gap between individual Soldier and Marine and their higher echelon units by providing 
commanders with a low resource, high impact small unit training tool. The COTS 
option, versus a slow and ponderous military contract system, allows for increased 
mobility (as all major DoD installations have at least 1 computer lab with LAN support), 
reduced cost at the unit and installation level, and the ability to tap into commercial 
intellectual reservoir. 

Although the use of live-five exercises remain the key vehicle for infantry squad 
training, using computer simulation is increasingly common in order to better prepare 
Soldiers and Marines for the live-fire events. In a February 2005 interview in Training 
and Simulation Journal, USMC Major General Jon Gallinetti, the Director for Joint 
Training, J7, U.S. Joint Forces Command Commander of the Joint Warfighting Center 
emphasizes how simulations, in conjunction with live training, can expand the live 
training footprint (TSJ, 2005). 

I don’t think anything will replace live training because I believe you need 
to do the live training to an extent, but I also believe that virtual and 
constructive training can certainly add to the complexity and to the 
dimensions of what you can include in an exercise. 

By using simulations as a precursor to live training, leaders can commit resources 
to training more complex events that are often neglected due to resource limitations. A 
virtual training environment that allows units to exercise and refine Standard Operating 
Procedures, rehearse reporting procedures and ground tactical movement can save leaders 
over thousands man-hours, ammunition, rations, and other resources that are typically 
used during the first 36 hours in the field when squads and platoons are preparing to 
conduct training. 
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In the same manner that NASCAR drivers use commercial games on a console 
platform to “proof the track” prior to a race (Emmons, 2003), virtual training can help get 
Soldiers and Marines “in the game” by focusing attention to the task at hand prior to 
going to the field. Squad simulations, if accessible and with a level of mobility, can also 
be used as a readily available rehearsal platform for deployed units (such as Marines 
afloat or Soldiers at a forwarded deployed area) to prepare for real missions. 

In today’s operations in Iraq for example, if an infantry squad receives a mission 
to conduct a cordon and search mission to seize illegal weapons caches, the unit will 
create from associated maps and UAV over-flight photos a terrain model in the sand from 
spent ammunition boxes and other available objects and talk through the mechanics of 
the unit’s movement through the urban area (Figure 1). Given enough time, the squad 
may conduct a live rehearsal using available buildings in the base camp to prepare for the 
mission. This train-up window will be condensed due to time available, resources on 
hand, ability to use blank ammunition in the cantonment area and numerous other 
restrictions. 



Figure 1. Terrain Model Briefing (Horne, 1998) 

With an accessible simulation tool that can be easily modified by the users, the 
squad can conduct a multiplayer mission rehearsal in an immersive virtual environment 
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that accurately represents the area of operations with a low resource commitment. Within 
this more robust virtual environment, infantrymen can use weapons without limitation. 
Squads can exercise fire control and distribution techniques, target effects, and rehearse 
movements in multiple environmental conditions in a relatively short period of time. 
Once the training is complete virtually, the leaders can focus on more complex tasks at 
the live training event thus improving the overall readiness of the unit. 

Virtual training can also be used to provide sustainment training, particularly for 
units with high personnel turn-over rates, in order to capture subject matter expertise and 
experience. As key leadership positions change within an infantry unit after any 
deployment, so does the experience base of the unit. Because of the current operational 
tempo of units today. Standard Operating Procedures, reporting procedures, tactics and 
techniques of conducting ground operations need to be quickly taught to new personnel 
and rehearsed collectively in order to be operationally effective. Often, the squad’s only 
chance to execute complex task rehearsals is during a major unit training exercise (NTC, 
JRTC or CALFEX) conducted prior to deployment to the area of operations. As a result, 
our infantry units only really conduct an initial “walk-through” of complex tasks (due to 
resource requirements), and really do not get a chance to refine their rehearsals until they 
are on foreign soil. With the availability of a small unit simulation platform, a 
redeploying unit can conduct complex exercises in a virtual environment with new 
personnel in order to pass on the “library of experiences” from departing subject matter 
experts with minimal time and resource cost. Conducting this “post-training” virtually 
will allow redeploying units to maintain a higher level of expertise, and minimize time 
and resource requirements of the redeployed units as the virtual environment can closely 
replicate the robust, complex scenarios the unit faced during their time in the area of 
operations. 

E. RESEARCH APPROACH 

The challenge that faces researchers in measuring the effectiveness of using 
simulations for collective training is that at the infantry squad level, much of the 
evaluation criteria are based on a trainer’s subjective evaluation of the unit. The infantry 
community utilizes the Army Training and Evaluation Plan 7-8 Mission Training Plan 
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(ARTEP 7-8-MTP) to evaluate performance measures of collective training at the 
infantry squad level. The ARTEP 7-8-MTP contains a series of performance checklists 
that the trainer uses to evaluate the readiness level of the training unit. A unit’s grade for 
a given task is solely dependent on the trainer’s interpretation of the performance 
measure from the ARTEP 7-8-MTP and his subjective evaluation of the unit 
performance. The evaluation scale also leaves little room for specific comparison: task 
performance is rated as passing (“go”) or failing (“no go”). Based on these factors, there 
is no quantifiable, absolute measure to determine if one unit is some percentage better 
than another. 

Eor researchers attempting to go further than the Army standard to measure a 
unit’s performance, the evaluation criteria become difficult to articulate or justify. Eor 
example, do we measure how quickly a unit shoots targets at the live fire range once the 
targets are presented or do we measure how many shots per target? This type of arbitrary 
evaluation lends itself towards far too many confounding variables. In our previous 
example, if we measured reaction time (first shot down range) - is this based on our 
unit’s training level or the marksmanship skills of one or two Soldiers? Could the better 
trained unit react slightly slower because of their more deliberate tactical movement 
based on a higher level of enemy threat analysis than the untrained unit that has 2 or 3 
great marksmen? Because of the current subjective methodology of measuring infantry 
squad performance, attempting to measure a simulation’s impact on collective training 
becomes even more clouded. 

Due to the difficulty in obtaining quantifiable performance data (and inevitable 
debate that would ensue based on any “quantifiable” performance measures that we could 
select), we took a different approach for our thesis research. Instead of trying to 
“reinvent the wheel” and create a new collective training measure for our thesis research, 
we set out to determine if commercial gaming software can be used by trainers based on 
the ability of the platform to replicate the audio/visual cues required to conduct collective 
training and the degree of user acceptance of the gaming platform as a training venue. At 
the core of games-based training, both of these questions are the key and essential 
measures of a system’s effectiveness. If a system cannot provide the cues necessary to 
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elicit the required response, then the system is not effective. Additionally, the users of 
the simulation must be able to look beyond the fact that they are at a computer 
workstation in order to accept the virtual environment as a legitimate training venue. 
Given these tenets of computer based training, we designed our research groundwork. 

The scope of this study examined the collective and leader tasks that are required 
for successful execution of the squad “React to Contact” battle drill in accordance with 
the ARTEP 7-8-MTP, and how those tasks could be trained with the use of a COTS 
gaming software while members of the unit are executing missions in a multiplayer 
virtual environment from individual desktop computer workstations. We kept the focus 
of the research to the “React to Contact” battle drill, because this task is the fundamental 
building block for the squad attack mission. Although we kept the scope of the 
experiment to the squad level, the findings from our research experiment could be may be 
applicable to the rifle platoon as well. 

We will discuss methods for properly integrating gaming technology into an 
existing training plan and game selection criteria in order to select the proper platform to 
meet the training unit’s needs and provide sufficient audio/visual cues to support user 
“buy-in” of the virtual training venue. We also explored ease of modification of a 
commercial product given nothing more than the commercially available mission editor 
in order tailor the virtual environments to meet specific unit requirements. The end state 
of this research study is to provide initial analysis on what collective skills COTS gaming 
software can be used to train at the infantry squad level, develop a training model 
recommendation for the integration of this tool into existing unit plans, and provide 
recommendations for product selection criteria. Given our end state, we framed our 
research questions as follows: 

• How should the infantry company and battalion integrate the use of COTS 
game exercises into their existing live, virtual and constructive training plans? 

• What are the qualities for COTS game selection to use as training tools for 
the infantry squad? 

• Will subjects accept the virtual environment as a legitimate training 
venue? 
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• Do users feel that a COTS game can provide the audio and visual cues 
required to conduct the “React to Contact” battle drill at squad level? 

• How quickly and accurately can a novice computer user create a virtual 
training environment given a COTS gaming mission editor? 


F. METHODOLOGY 

In order to properly address our research questions, we outlined a methodology 
roadmap to focus our efforts. Our roadmap included the following steps: 

• Clearly define the critical collective tasks required for execution of 
infantry squad “React to Contact” battle drills in accordance with ARTEP 7-8 
Drill. 

• Select a game platform that can: 

• Provide sufficient audio/visual cues that can drive a collective 
infantry training event. 

• Allow for easy modification of the virtual environment to tailor 
towards meeting specific unit training needs. 

• Can run on hardware specifications that are not commonly found 
on computers at installation laboratories. 

• Can be easily learned by the test subject and can support up to 50 trainees 
in a multiplayer session over a local area network. 

• Map the tasks, audio/visual cues, and environment of the “React to 
Contact” battle drill to a COTS game platform. 

• Create a survey that will capture user’s level of platform acceptance, 
experience and any other relevant data required to determine if the COTS game is 
a suitable training venue. 

• Conduct pilot and field experiments to answer the research questions using 
a variety of different users, and capture all relevant qualitative data that results 
from the use of the COTS gaming software. 
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Our research was conducted over four phases at two locations: The Naval 
Postgraduate School, Monterey, California and the United States Infantry Center at Fort 
Penning, Georgia. The four phases were broken down into the following: 

• Phase I: Analysis of key collective tasks required for execution of infantry 

squad battle drills 

• Phase II: Mapping the key collective tasks to the COTS game platform 

and execution of the pilot study 

• Phase III: Execution of the test experiment 

• Phase IV: Analysis of the findings and recommendations 

During Phase I, we conducted a task analysis of the “React to Contact” battle drill 
utilizing the ARTEP 7-8 Drill as a guide to determine the critical collective tasks that are 
required for successful battle drill execution. We also selected the COTS game platform. 
Delta Force: Black Hawk Down - Team Sabre™ by Novalogic™, Inc., as it provides a 
common game interface for our test subjects that was not complex to operate, accurately 
modeled current infantry weapons, provided a multiplayer environment and did not 
require a high-end graphics card or processor . Using the mission editor, two preliminary 
virtual environments were created for the conduct of the pilot and field tests. To support 
the training scenarios, we created eight mission orders with operational graphics for our 
subjects in order to drive the planning and squad operation orders process. We 
completed this phase by identifying the key collective/leader tasks required for infantry 
battle drill execution, selecting the game platform, and completing our virtual training 
venues for the execution of the “React to Contact” battle drill. 

Phase II was the conduct of our local pilot experiments using test subjects from 
the Naval Postgraduate School and Defense Eanguage Institute to measure the 
effectiveness of the game platform to model (or replicate) the cues required for leaders 
and unit members to execute missions during a battle drill. The results of our pilot 
experiments were used to determine the “alpha” tasks (those to be measured with during 
the unit level experiment) from the total task list. The “beta” tasks, those in which the 
game did not accurately provide cues for subject execution, were used as our preliminary 
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starting point for analysis on what tasks or skills the game cannot be used to train. 
Additionally, we used the pilot studies to refine our subject surveys, rehearse timings of 
key events for the conduct of the field test, and ensure the virtual environment that we 
created with the Delta Force: Black Hawk Down - Team Sabre™ game mission editor 
provided enough of challenge to exercise the trainee’s teamwork without being so 
difficult that we would alienate our subjects. To prepare for the field test, we traveled to 
Fort Benning, Georgia in order to determine the hardware and software capabilities of the 
test site, coordinated for the scheduling of test subject support (Second Lieutenants 
attending the Infantry Officer Basic Course - lOBC), and brief the Fort Benning 
Command Group on the task, purpose and end state of our thesis research. We concluded 
Phase II by creating an initial list of “alpha” and “beta” tasks, completing the 
coordination with lOBC and finalizing our resource requirements for the field test. 

Phase III was the conduct of the unit experiment. Through coordination with the 
command group of lOBC, we randomly selected two 10-11 man rifle squads as our test 
subjects (those who would receive the games treatment) and two rifle squads as our 
control group (those who would not receive the games training). Based on the lOBC 
training plan, we elected to use the games as an additional training event versus an in lieu 
of training event. The subjects had completed initial individual skills training which 
included weapons marksmanship, land navigation, and drills for crew served weapons. 
The subjects had received their classroom block of instruction on the “React to Contact” 
battle drill, but had not yet conducted any formal field training to apply what they had 
learned in class. 

In discussions with lOBC, we decided to conduct our games-based training 
experiment after the completion of the classroom block of instruction and prior to the 
lOBC field deployment for the conduct of their first live fire squad “React to Contact” 
exercise. The control groups received no additional training other than the classroom 
block of instruction prior to the live fire exercise. Each test group underwent an eight 
hour training event with the COTS game that included an individual familiarization 
exercise, a unit familiarization exercise and six or seven mission exercises (the difference 
was due to time constraints) with operational graphics, maps and overlays (from the 
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game) provided to the unit. The leaders and key members of the squad were selected at 
random. We used the ARTEP 7-8 Drill for the conduct of an after action review (AAR) 
after completion of each exercise, and subjects completed a comprehensive survey prior 
to, during and after the conduct of the exercise to capture all relevant data on prior 
experience in conducting infantry ground operations, computer game use, individual 
training assessment, and assessment of the game platform as a viable training venue. 
Following the conduct of the experiment, we deployed to the live fire field site with both 
the test and control squads to observe qualitative data on the unit’s performance and 
complete our test and control group survey and after action review as well as complete 
personal interviews with the lOBC leadership for assessment and recommendations for 
COTS games integration into unit training plans. 

Phase IV began with the analysis of the data and thesis document production. In 
addition, we had the opportunity to conduct a very limited test on the game platform’s 
mission editor to determine ease of use and time required to create a robust visualization 
given an overhead UAV photo. The intent was to determine if the game can be modified 
quickly enough, by a trained user, to be effective for forward deployed units as a mission 
rehearsal platform. For this task, we utilized a U.S. Air Force Academy Cadet on an 
internship at the MOVES Institute, who had very limited computer experience (games or 
otherwise), to create three virtual environments based given only a cursory introduction 
to the mission editor, three current UAV overhead photos of towns in Iraq and 
Afghanistan, and a computer workstation. We concluded this final phase by completing 
our findings and providing recommendations for future thesis research work. 

G. THESIS ORGANIZATION AND TABLE OF CONTENTS 
Chapter I: Introduction 
Chapter II: Background 

Chapter III: Conduct of metric selection for use of experiment selection (Phases T 
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Chapter IV: Conduct of the Experiment (Phase III) 

Chapter V: Analysis of the results (Phase IV) 

Chapter VI: Recommendations and Conclusion 
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Appendix A: Operation Orders And Graphics 
Appendix B: Subject Surveys 
Appendix C: FM 7-8 “React to Contact” Battle Drill 
Appendix D. Institutional Review Board Documents 
Appendix E. Experiment Set Up Using The Mission Editor 
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II. BACKGROUND 


A. FRAMING THE PROBLEM 

The core of our research is the examination of whether COTS games can be used 
by trainers to make our infantry squads better prepared for diverse and complex 
situations, more effective in communicating tactically and overall a more lethal combat 
force. To this end, we broke down our hypothesis into three key areas for our literature 
review: 

• What defines an effective team? 

• What impact do simulations have on improving team performance? 

• What skills can a COTS game train? 

With a goal of improving overall team performance, we needed to examine what 
makes an effective team work, and design our experiment with an environment that 
supports, in part, tenants of a successful team. Next, we had to explore whether 
simulations, of any type, can be used successfully to train teams. Finally, we had to look 
to previous work on the leveraging of COTS software by the Department of Defense in 
order to identify advantages and disadvantages as well as techniques for using 
commercial games. Exploring these three areas in the existing literature provided some 
direction for our research, and ideas for implementation of our experiment. 

B. DEFINING THE EFFECTIVE TEAM 

1. Factors of Effective Teams 

Infantry squads, much like any small unit that conducts high risk tasks 
(paramedics, SWAT teams, etc.), need to perform at peak efficiency in order to conduct 
operations while reducing casualties and damage to critical infrastructure such as 
churches and schools. Understanding the factors that define an effective team and 
developing tools that can help teams achieve goals are keys to ensuring success. 

During the formation of any team, there are four distinct phases that a group of 
individuals goes through to create an effective force: forming, storming, norming and 
performing. Forming consists of the group of individuals coming together to solve a 
problem or work on a given task. The formation of the team can be either self-directed or 
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dictated by a higher headquarters. This phase usually ends with the group identifying 
goals and initially selecting group members to execute each task. Next, the team goes 
through the storming phase. This phase is characterized by disruption as the group 
debates techniques for goal accomplishment, selection of leadership and distribution of 
workload. As the team finds solutions for these areas, they enter the norming phase. In 
the norming phase, the group has settled on leadership roles, accepted the methodology 
of task accomplishment and workload distribution. Finally, the team enters the 
performing phase where the unit works together towards the common goal through 
effective communication, mutual support and accomplishment of subordinate tasks that 
contribute to the overall mission (Sheard and Kakabadse, 2002). 

Sheard and Kakabadse (2002) explored the contributing factors of effective teams 
and identified which factors were crucial at each stage of team development. Observing 
teams from a multinational engineering company through the forming, storming, norming 
and performing stages, the authors defined the following nine factors of effective teams: 

• Clear goals that are understood by all. 

• Priorities aligned with subordinate team’s duties and responsibilities. 

• Roles and responsibilities that are agreed to and clearly understood. 

• Self-awareness that in turn drives team behavior. 

• Leadership that serves as a catalyst for dynamic changes. 

• Group dynamics defined by a social system (or chain of command). 

• Communications defined by open dialogue. 

• Context of orders influenced, but not directed by organization. 

• Infrastructure supported by the organization. 


The author’s “team landscape” (mapping the nine factors of team effectiveness 
over the four team forming stages) provided us with ideas for groundwork for creating an 
experimental environment that would support our test subjects. Our goal was to 
minimize disruptions to the team by providing a test environment that would support the 
subjects, thereby focusing the effort on evaluation of the COTS software. 
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Page and Donelan (2003) focused more on the individual roles team members for 
their factors of effective teamwork. Creating elements of a psychological contract (a 
contract based on common goals and values), the authors identified the need to assign 
responsibilities and accountability to individuals, foster accurate and timely 
communication, and have leaders check subordinates for clear understanding of orders. 
We ensured that the elements that the authors identified were present by directing (in the 
operations order) our subjects to conduct back-briefs and inspections of individual team 
members to make sure the mission and intent were clearly understood. We also 
questioned key leaders to ensure that mission essential tasks were delegated. With this 
delegation of the tasks, we also monitored the leaders to ensure an inspection mechanism 
was present in order to ensure delegated tasks were tracked. These control measures 
were not unique to our experiment, and are common in any ground force chain of 
command. Our challenge was to replicate a “higher headquarters” for our experimental 
group in order to create a sense of the unit belonging to a greater force, and keep the fact 
that our subjects were, in fact, operating independently in a virtual environment 
transparent. 

2. Challenges of Measuring Team Performance 

As previously discussed, neither the Army nor Marine Corps has true quantifiable 
performance assessments for infantry squads. The evaluation criteria utilized in both 
organizations consist of performance measures and task checklists for mission 
accomplishment, but the degree to which the unit successfully performs the action rests 
on the evaluating training officer or NCO. As a result, attempting to produce a 
quantifiable team performance measure for our experiment presented a challenge. Jack 
Zigon (1998), President of the Zigon Performance Group which specializes in customer- 
performance management and team performance measurement systems, describes 
difficulties of team performance measurement and provides some techniques for 
overcoming this challenge. Some of the challenges that he addressed are: 

• It is not always obvious what results should be measured 

• Even if you know what to measure, it is often not clear how the 

measurement should be done 
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• Teams are made up of individuals, thus the measurement must be done at 
both team and individual levels 


Zigon (1998) goes on to present different techniques for measuring team 
performance with groups who have quantifiable goals (such as retail sales departments), 
and describes a technique for extracting qualitative data from groups without concrete 
measures. Because of the difficulty in designing a reliable measurement system for 
infantry squad performance, the latter technique presented us with some ideas to capture 
meaningful qualitative data from our test subjects. He notes: 

Not everything can be measured with numbers. Trying to quantify 
everything sometimes results in meaningless measures. Good measures 
are those that can be verified by someone else and are 
observable... Descriptive measures use words to evaluate the 
accomplishment and identify who will judge the performance and what 
factors they will be evaluating. 

Given this recommendation, we elected to include the training officers and NCOs 
from lOBC for the conduct of the experiment. Since we ultimately wanted to determine 
if COTS games can be used by training units, we felt that the performance evaluation of 
the team should remain with the test subjects’ current instructors. This method allowed 
for consistency of evaluation, and provided us with an accurate current training 
assessment (versus our limited assessment for the 6-7 hours we worked with the test 
groups). Finally, Zigon (1998) recommends the use of feedback system to capture the 
data of the team’s performance. He describes the steps for data collection that include 
determining what data to collect, feedback sources, data collectors and the appropriate 
use of aggregation of data. The author’s description of this feedback system helped us 
begin to frame our subject survey methodology to capture data from our experiment. 

3. Using Self-Assessment as a Measure of Team Performance 
We realized that we needed to gather self-assessment data from our subjects in 
order to measure how our subjects feel that using the COTS game would affect their unit 
performance. Bateman and Wilson (2002) describe an experiment where they used 
individual self-assessment as an audit tool to determine team effectiveness. Using a 
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Likert scale self-assessment questionnaire, the authors examined 400 subjects from 37 
healthcare professional teams in order to measure team effectiveness. They limited the 
areas for assessment to measurements to: 

• Team synergy: A sense of purpose which is shared among the team 
members 

• Performance objectives: There are clear performance objectives which 
have been established by the team which are monitored on an ongoing basis 

• Skills: Team members are adequately trained and competent to do their 
work 

• Use of resources: All resources are used effectively 

• Innovation: The degree to which the team looks for ways to improve 
products and systems 

• Quality: Degree of customer awareness and standards are identified and 
monitored 


By using this individual self-assessment tool, the authors ran correlations against 
quantifiable team measures to determine effectiveness of the technique. They found a 
high correlation between the self-assessment and the actual team performance (based on 
organizational measures such as customer satisfaction). Additionally, the subjects used 
the results of the anonymous self-assessment to identify areas for improvement in order 
to make the overall organization better. We adopted this methodology of self-assessment 
as a measure of team performance and user acceptance for our experiment. We felt that 
the degree to which our subjects responded to acceptance of the platform, effects of the 
training event on individual and team performance, and ability of the COTS game to 
provide audio/visual cues to drive human interaction would provide us with the insight to 
begin answering our research questions. 

C. LEARNING, TEAM PERFORMANCE AND SIMULATIONS 
I. Effecting Team Learning and Training Techniques 

Although the literature identifies training as a key and essential component of 
team effectiveness, exploring how teams and individuals learn is fundamental to 
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understanding training techniques. “There are many training methods available; some are 
more effective than others. To understand why one method may be more effective than 
another, it is important to have a basic understanding of how people learn.” (Read, 1996). 
Effective learning, as defined by the authors, includes active participation by the trainee, 
positive reinforcement of actions correctly performed and clearly understandable benefits 
of the experience by the participants (Read, 1996). 

Feinstein, Mann and Corsun (2002) continue the examination of learning by 
noting that learning “exists when the learner processes information in an active and 
immersive learning environment.” (Feinstein, 2002). With this in mind, we strove to 
ensure that our virtual environment provided an immersive scenario populated with 
adequate events to illicit active participation by all participants to support individual and 
team learning. 

Although initial training of the officers attending lOBC includes lectures, the bulk 
of the infantry training events includes active participation by all team members in as 
robust an environment as resources permit. Select trainers are designated as 
observer/controllers (OC) and serve as the subject matter experts (SME) to provide the 
feedback for members of the training team. Through the use of the after action review 
(AAR), OCs make recommendations for individual and team improvement, and provide 
the direct link from training to the real world. The shortcoming for infantry training 
occurs when resources are not allocated in order to meet the requirements. The question 
is whether computer simulations can bridge the gap when people, time and resource 
deficiencies prevent the conduct of robust, immersive training exercises. 

2. Using Simulations for Training Teams 

For infantry training, nothing will replace going to the field and conducting live, 
realistic, collective training on terrain similar to where infantrymen can expect to fight. 
Major Wilfred Rodriguez, branch chief for the Dismounted Battlespace Battle Fab at Fort 
Benning, Georgia, discusses these challenges in Infantry Magazine, Fall 2003 
(Rodriguez, 2003). 
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Training areas are busier, and are not as plentiful as in the past. Ball and 
tracer ammunition and grenades are scarce. Missiles are expensive, and 
small arms rounds’ usual priority of issue is to support marksmanship. 

Given today’s reality of resource constraints and limited time, alternative training 
techniques must be pursued. Additionally, better use of live resources could be achieved 
by conducting preliminary virtual training prior to a live event. To this end, the author 
outlines what he identifies as requirements for an infantry-centric simulation tool to 
bridge the gap when resources are unavailable, and to make the most of live fire 
exercises. Major Rodriguez states that a “simulation is needed that gives the infantryman 
training value in offensive and defensive scenarios in the types of terrain that we 
(infantrymen) will fight”. He describes the need to get “whole unit interacting together in 
a virtual battlefield” that replicates the venue and threat of today’s enemy force 
(Rodriguez, 2003). 

Virtual simulations are a potential solution to training resource scarcity, 
integrating highly lethal and costly combat multipliers and developing the 
skills to use them. These virtual simulations could serve as an excellent 
training gate to gain a certain level of proficiency before units execute 
costly, rare and dangerous live training. These same virtual simulations 
make mission rehearsal for far flung or quickly developing contingency 
operations possible. 

As Major Rodriguez correctly notes, the human cost - the cost associated with 
high risk training - also contributes to the need for simulation use. In a November 2001 
interview with National Defense, Brigadier General Stephen M. Seay, former 
Commanding General of the Army’s Simulation, Training and Instrumentation 
Command, expands on the risk cost savings (Kennedy, 2001): 

Soldiers still have to learn to deal with the extreme risks of real-life 
combat, and live fire is needed for that, but simulations let them 
experience something that’s very close to the real thing, without any 
danger. 

General Seay notes that cost savings can have a significant impact, beyond risk, to 
include resources required for live events and training readiness. He notes that “in 
simulation, when you’re finished with a drill, you can go back and do it again in a matter 
of seconds” and not incur the costs of field-feeding troops, rest plans, transportation and 
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equipment maintenance (Kennedy, 2001). By having the opportunity to repeat complex 
scenarios relatively quickly in a virtual environment, units can refine SOPs and 
techniques that allow them to focus on more challenging conditions when deployed to a 
live training scenario. Repeatability of scenarios allows units to build a “library of 
experiences” that increases proficiency and expands footprint of expertise throughout the 
entire unit. 

In commercial business, companies are using computer simulations tools to build 
their subject matter expert (SME) base with preliminary success. Much like the military 
model, businesses are dealing with high personnel turnover, and as a result, their 
management teams struggle with maintaining a high level of expertise with limited 
resources. One tool that is being used is computer-aided vision (CAV). CAV is a low 
cost computer simulation that can allow business leaders to conduct operations in a 
virtual venue and provides “a realistic and engaging vehicle to stimulate managers to 
reconsider their ways of doing things, and perhaps adjust their mental models” (Winch & 
McDonald, 1999). With the ability to repeat scenarios under different conditions, 
managers can build upon their experience base. Although CAV cannot be used to predict 
real-world outcomes, it can provide a venue to allow managers to exercise critical 
thinking techniques that “could, at least partially, overcome the disadvantages” of a 
smaller SME base (Winch & McDonald, 1999). Much like the CAV model, simulations 
can provide infantrymen with the same non-attribution type venue to build an experience 
base prior to conducting live exercise which will reduce cost, and increase efficient use of 
real resources. 

3. Methods for Using Simulations in Team Training 

Although computer-based simulations can help units build virtual experiences, 
they must be used as a collective training tool (with the entire team participating) to truly 
improve team effectiveness. Eeinstein (2002) correctly notes that a shortfall to single 
participant computer-based training is the absence of interpersonal learning that is 
associated with role-play training common in live exercises. In addition, simulations, by 
themselves, must not be viewed as a panacea. Simulations are not in itself the trainer, but 
the tool used by leaders to enhance performance of the team (Bell, 1999). 
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To be effective, trainers considering use of computer simulations must clearly 
define the task, purpose, intent and expected goals of training prior to conducting any 
exercise. Subsequent to defining the tasks, a task analysis must be conducted to 
determine the individual and crew tasks that are required to conduct the collective event. 
With these two requirements met, trainers can determine how best to maximize this 
training tool. Schlager (1994) described a technique for setting the groundwork for 
virtual environment use. In his article, he recommends that trainers determine the task 
selection criteria by outlining the training constraints (tasks that can and cannot be done 
in VEs), the impact of the event, and expected learning outcomes. Tasks must consider 
participants, environments and the knowledge, skills, and abilities (KSAs) required, while 
clearly outlining the tasks, steps for conducting the task, causal links, ordering and roles 
of the team members (Rickels, 1999). It is equally important for the conduct of the event 
that the instructional requirements (lesson plans and feedback mechanisms) and outcome 
requirements (partial task performance) are clearly defined. 

Ultimately, the cost effectiveness of the use of these types of tools must be 
conducted to determine if investment is worth the outcome (Schlager, 1994). For 
example, if the use of a fixed computer simulation facility requires more man-hours to 
train Soldiers on its use than it can be used for training or costs hundreds of thousands of 
dollars for trainers to operate and maintain, is the cost worth the benefit? Alternative 
solutions must be considered in order to leverage the benefits of simulations without 
expending the types of resources these products were designed to conserve. 

For the conduct of a simulation exercise, the issue of distributed applications 
versus centralized training location needed to be addressed. Specifically, do teams 
perform better centrally located or can teams gain the same benefits from remote sites? 
Potter and Balthazar (2002) examined the structure and performance of face-to-face 
(FTF) and virtual or computer-mediated communication (CMC) teams to determine 
performance measures and differences between the two media. The authors noted that 
findings from previous experiments included increased performance of CMC teams with 
regards to candor of personnel when discussing difficult issues while FTF teams 
demonstrated increased acceptance (or team loyalty). The authors decided to conduct an 
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examination of CMC and FTF team structures and include the individual personalities to 
measure performance. Ultimately, the authors concluded that predicting team 
performance, whether a CMC or FTF team, relies more on the team interactive style than 
their media. Bell (1999) examined the use of Distributed Mission Training (networked 
simulations for team training). In his research, he found that participants participating in 
full spectrum networked simulations (F-15 fighter simulators) exercises as part of Road 
Runner’98, in which flight simulators at several distant locations, demonstrated 
“considerable improvement on both the process (communication quality and frequency) 
and product (numerical flight data) measures.” (Bell, 1999). Potter, Balthazar and Bell’s 
work is important as it leaves running a distributed application open as a possible 
technique when leveraging commercial gaming technology for training. Distributed 
application could reduce cost for units conducting this training by limiting the up-front 
cost of a fixed venue; however, interpersonal learning that Feinstein (2002) noted 
previously could be impacted without effective communication tools available to all 
players. 

D. USING COTS GAMING SOFTWARE FOR TEAM TRAINING 

I. Emergence of PC Based Simulations 

As computing power has increased, simulations once reserved for mainframe 

computers are now available for the personal computer. Additionally, network 

architectures, local area networks and unit level intranets allow for the use of multiplayer 

simulation exercises to be conducted at a unit level without the scheduling of a fixed 

facility (Rodriguez, 2003). This flexibility of training, only recently enabled by 

technology advances, opens the door for the leveraging of low level simulations by unit 

leaders to conduct task specific training events. The commercial gaming industry, in an 

ever increasing race to remain competitive, has created very robust military multiplayer 

games that have potential for training collective tasks at the infantry squad level. Warren 

Katz, CEO of MaK Technologies (a commercial game developer) notes in a November, 

2000 interview in Training and Simulation Journal that “improvements in 3-D image 

generation on the PC and the speed of the internet” have increased the military’s interest 

in the use of video games as training tools (Erwin, 2000). He goes on to say that “PC- 

based video games have come so far in the quality of imagery, that they rival multi- 
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million dollar Defense Department simulators” (Erwin, 2000). Although Mr. Katz does 
not address ability of video games to match the complex algorithms that make up the 
underlying architecture of DoD level simulations, he is correct in his observation that use 
of COTS gaming software remains a key area of interest for DoD. 

General Kevin P. Byrnes, former Commander of the U.S. Army Training and 
Doctrine Command at Fort Monroe, Virginia addressed the need for wider use of 
computer games-based simulations in training in the August/September 2005 issue of 
Training and Simulation Journal (Anonymous, 2005). 

The best use of simulations lies now with leader-type training exercises 
using small laptop driven vignettes where small unit leaders anywhere are 
immersed in a combat situation in the contemporary operating 
environment. After a few minutes, they start feeling as if they are really in 
it. 

In General Byrnes’ comments about general simulation use, he addresses three 
key areas in which COTS games may be utilized: keeping cost low by making the 
simulation able to run on a laptop, maintaining an engaging learning environment for 
users, and vignette driven design. When determining selection criteria for the COTS 
platform used in our experiment, we modeled our top three screening factors in a similar 
manner: computer system requirements, immersion and user ability to easily modify the 
“vignette.” In particular, the ability of the trainer to easily modify the scenario could 
allow a COTS game platform to be used as not only a tool for home station training, but 
as a mission rehearsal platform for forward deployed Soldiers or Marines aboard ship. 

The Marine Corps, due to the training resource limitations while afloat, has 
embraced COTS gaming technologies, and is aggressively looking at ways to exploit 
games as a virtual training asset for all military occupational specialties (MOS). The 32"‘* 
Commandant of the U.S. Marine Corps, General James L. Jones, in an interview in 
National Defense stated that commercial computer games have “great training value” for 
the force (Erwin, 2000). 

There is a Squad Eeader’s course where a Squad Eeader can stand up in 
front of a giant screen and actually run through a particular scenario (from 
urban to desert terrain). The computer picks up (the Squad Eeader’s) 
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command, as if he were talking to a real squad...When he is engaged by 
an enemy force, he can maneuver his squad, and they react on the screen. 

General Jones identified the importance of integrating live, virtual and 

constructive simulations to achieve a balanced training approach, but felt that “simulation 

can do an awful lot for learning the basics” and thus make live training more productive 

(Erwin, 2000). By providing an environment to learn cognitive skills, leaders can use PC 

games to help train critical thinking and problem analysis skills prior to one live round 

being fired. General Byrnes goes further to address the potential of PC games to provide 

a venue where cognitive skills are exercised (Anonymous, 2005). 

It (a PC simulation scenario) poses questions that (leaders) really have to 
think through. The more we get combat veterans to help us redesign the 
vignettes with the realism they experienced in Afghanistan or Iraq, the 
better quality the training is. 

Although there are DoD simulations where modifications can be made based on 
end user requirements, they typically require from weeks to years to develop, and 
necessitate the use of a contractor to adapt. In contrast, the commercial game industry, in 
an attempt to extend the life of their product, has begun to embed level (or mission) 
editing software to allow consumers to create unique environments. By placing the tool 
directly in the hands of the end user, COTS gaming software can be used to develop 
specifically tailored training venues to meet specific unit requirements. Additionally, as 
General Byrnes points out, units could use Soldiers or Marines with recent combat 
experience to develop scenarios for training using COTS software. 

2. Tactical COTS Game Use within DoD 

The Defense Advanced Research Projects Agency (DARPA) has funded a 
project, DARWARS, which utilized the COTS game market for the development of 
training systems based on the contemporary operating environments of Afghanistan and 
Iraq. The first of these efforts. Ambush!, is a PC-based multiplayer simulation tool 
designed to provide a venue where squads can learn the challenges and dangers of 
wheeled convoy operations (Walker, 2005). The game provides scenarios where users 
learn how to identify an lED (Improvised Explosive Device), and “it trains convoy 
Soldiers to think and act like a team” (Walker, 2005). Based on the COTS game 
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Operation Flashpoint by Bohemia Interactive, Ambush! is currently used by Army units 
at Fort Lewis, Washington and Fort Hood, Texas as well as Marine Corps units stationed 
at Twentynine Palms, California. “'DARWARS' goal was to create a scaleable 
architecture...that could be used to train small groups of people in team thinking” 
(Walker, 2005). Ralph Chatham, DARPA program manager for DARWARS, explains 
that the game is “not teaching how to shoot or to drive; it’s teaching the cognitive” 
(Walker, 2005). By providing contemporary scenarios, DARPA believes that small 
teams can learn the critical thinking skills that combat operations in Iraq or Afghanistan 
demand, but results on the effectiveness of Ambush! have not been presented. 

MAJ Carl Jaquet, Director of the United States Military Academy Warfighting 
Simulation Center (WARCEN), noted that the Department of Military Instruction (the 
tactical training arm of the academy’s education program) has been using PC-based 
games for cadet training for quite some time. WARCEN has expanded the breadth and 
scope of team training by using a distributed multiplayer approach and leveraging 
commercial off the shelf (COTS) products as alternatives to high end DoD simulation 
tools. WARCEN developed training scenarios in which infantry squads conduct combat 
operations using level editing software from the COTS game Operation Flashpoint™, 
created by Bohemia Interactive Studio'^'^, and America’s Army™, developed at the 
MOVES Institute at the U.S. Naval Postgraduate School. Because of time and laboratory 
resource constraints, MAJ Jaquet set up the multiplayer exercises for cadets to either run 
from their dorm room or meeting collectively at a fixed location. He noted that the 
academy was expanding its WARCEN operation to support up to 130 cadets in an online, 
multiplayer scenario in order to support team training from the infantry fire team to a rifle 
company sized element (2004). During the initial experiments conducted by WARCEN, 
MAJ Jaquet observed anecdotal improvement to the cadets’ team performance on 
cognitive tasks such as command, communication and control of key weapon systems 
during the academy’s annual summer field tactical exercise (Jaquet, 2004). 

In addition to the work conducted at USMA, the Army has looked at several other 
COTS games for conducting small unit “cognitive” training. At Eort Benning, Georgia, 
captains attending the Infantry Captain’s Career Course (ICCC) have used Full Spectrum 
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Command, a PC game where the players command a light infantry company in a combat 
scenario. William Fisher, president of Quicksilver Software which developed the game, 
stated that “This (Full Spectrum Command) is not a skills trainer. We are training your 
mind. You’re not holding a gun” (Pappalardo, 2004). Although the game was well 
received by the staff at ICCC, the debate over the effectiveness of the tool remains to be 
resolved. In another example, the Army has used SpearheacF^ (a COTS Ml tank 
simulation game) as a virtual venue to train armor crews on armor maneuvers as well as 
artillery fire control. With SpearheacF^, under a $750K contract awarded from PEO- 
STRI, the Army worked with the game developers Zombie Virtual Reality Studios'^’^ and 
MaK Technologies'^’^ to modify the COTS game platform for HLA (High Level 
Architecture) compliance in order to allow scenarios driven from the game to be sent 
digitally to the Army’s Force XXI Battle Command, Brigade and Below and Control 
(FBCB2) platform (Erwin, 2000). By requiring HLA compliance, the Army was able to 
absorb the robust virtual environments from the PC-based game for crew training, and 
allow their actions to be sent to the FBCB2 system in order to extend the training 
footprint to the maneuver commanders and their staffs. Spearhead has been well 
received by users at the Army’s Fourth Infantry Division, but measures of effectiveness 
for the tool are unclear (Erwin, 2000). In both examples, user acceptance of the platform 
by users, ability of the game to replicate audio/visual cues required by a human operator 
to conduct complex operations and a task analysis of skills that can and cannot be trained 
with a COTS game are not discussed. 

As with the Army, the Marine Corps has explored the use of COTS software to 
train Marines. The first of these efforts was the training individual team members with 
first-person-shooter games such as Doom'’'’^ and Tactical Decision Games (TDG) 
simulations used for developing platoon commanders such as Close Combat Marine™. 
In an effort to train platoon commanders to combat terrorism, the Marine Corps helped in 
developing the PC-based COTS game Red Phoenix™. Created by Atomic'’''^, Inc., Red 
Phoenix™ is loosely based on Larry Bond’s novel about a North Korean invasion of 
South Korea (Peck, 2004). In Red Phoenix™, up to eight players can command a platoon 
of Marines and use most combined arms assets available to a rifle platoon including 
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indirect fire and close air support. What makes Red Phoenix™ unique is the 
psychological computer modeling of civilians on the battlefield (COB). The leader’s 
actions, either positive or negative, in the game scenario can either cause the civilians to 
support the U.S. actions or create an environment where COBs become disenfranchised 
with the Marines and support terrorist cells. Michael Woodman, program manager for 
Marine Corps Systems Command whose office co-developed Red Phoenix™ with 
Atomic'^’^, described the positive impact of games-based training as providing a venue 
where Marines can learn the six C’s: command, control, communications, cooperation, 
coordination and cognition instead of teaching a specific task such as marksmanship 
(Peck, 2004). As with the Army’s Full Spectrum CommancF^, the military use of Red 
Phoenix™ is not as an individual skills trainer, but as an environment where players can 
develop cognitive .skills. 

3. Innovative COTS Game use Beyond Maneuver Training 

In addition to Red Phoenix™, the Marine Corps developed the next generation of 
Close Combat Marine™ entitled First to Fight™. Unlike more robust multi-million 
dollar DoD level simulations. First to Fight™ was developed by extending an available 
COTS product at a cost of $800K. The Marine Corps took an innovative approach to the 
basic tactical simulation training with this new PC-based game. Identifying the growing 
concern of illicit drug use within their ranks, the Marine Corps developed a module 
within First to Fight that allowed a player to conduct the mission while under the 
influence of illegal narcotics (Peck, 2004). By severely limiting the player’s ability to 
control his squad, presenting computer-operated squad members who disregard orders 
and act dangerously while in a combat scenario, the Marine Corps’ intent of the game 
module was to reinforce the dangerous consequences that drug abuse can have on an 
infantry squad deployed to a combat zone. Although the drug abuse awareness module 
can be turned off and players can use First to Fight as a virtual venue for maneuver 
training, the fact that the Marine Corps Systems Command thought outside the box for 
different uses of COTS technology sets a precedent. 

In another example of innovative leveraging of the PC-game genre, the Army 
Research, Development and Engineering Command, along with business partners Warner 
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Brothers On-line'^’^ and the Institute for Creative Technologies'^’^, for $450K developed a 
training platform to improve Soldier-level situational awareness while deployed to Iraq or 
Afghanistan. Every Soldier a Sensor was designed to teach Soldiers how to use their 
observations and judgments of the environment and turn the findings into actionable 
intelligence. Presented as a first-person-shooter game, the players gain points by 
collecting information and spotting hazards, such as Improvised Explosive Devices 
(lED), instead of eliminating terrorists (Peck, 2005). U.S. Army Eieutenant Colonel 
Yvette Hopkins, Division Chief for the U.S. Army’s Actionable Intelligence Task Eorce, 
said of Every Soldier a Sensor (Peck, 2005). 

If I’d had this tool when I was doing the train-up for all the guys we 
headed out to Mosul, it would have been great. It’s getting people into the 
mindset. I can get that from this game, and trust me. I’m not a game 
person. 

In both the examples, innovative use of PC-based or COTS software for other 
than tactical exercise allowed leaders to reach a broader to training audience by providing 
an engaging learning environment that was accessible and cost effective. 

4. Mitigating Financial and Time Costs with COTS Games 
Understandably, certain military simulations such as JANUS, OneSAE, and BBS, 
an Army Brigade/Battalion Battle Simulation, are expensive to develop, operate and 
maintain. The required level of detail for algorithms, scale of the exercises and entities 
involved, and interoperability issues drive not only financial costs, but also costs 
associated with product development time and user learning time. Due to these costs, 
availability of simulation products becomes resource intensive requiring management at 
the battalion or brigade levels. While it must be made clear that COTS gaming software 
is not meant to replace, but to complement high-end simulations. Potentially they can 
provide any unit access to a simulation where Soldiers and Marines can train complex 
tasks in an environment with unlimited resources. 

Eurthermore, leveraging COTS software can provide military simulation 
developers with access to a wide range of tools to recommend to the field with a lower 
overhead cost than developing a system from scratch. Colonel Matt Caffery, Professor of 
War Gaming at the Air Command and Staff College and senior reservist in the Air Eorce 
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Research Lab’s Information Directorate described this trend in an interview in a 
November 2003 National Defense (Peck, 2003). 

People in the Department of Defense will go to commercial designers and 
say ‘your game is almost what we want.’ Instead of paying millions to 
design it ourselves, maybe you can customize it for much less. 

Leaders in the Department of Defense are not the only ones who are observing 
this trend. The commercial gaming market has become more aware of the uses of their 
products beyond the entertainment industry. Mark Herman, president of Booz Allen 
Hamilton and Breakaway Games, notes that senior leaders are just as concerned with 
development time of a “good enough” simulation as they are with financial cost (Peck, 
2000 ). 

There is interest in commercial games, because the senior military guys 
are saying ‘I can’t wait two years (for in-house simulations). If I can go to 
CompUSATM and get a game that I can get some insights and answers out 
of, why can’t we do that?’ 

Both COL Caffery’s and Mark Herman’s observation reflect a risk management 
approach to software development where the initial start-up and development costs are 
absorbed by the commercial market. By avoiding this huge initial investment (in the case 
of SpearheacF^, development costs exceeded two million dollars), DoD can make a 
much smaller investment by extending the COTS product for its use (in our SpearheacF^ 
example, the Army invested $750K for the modified game platform) (Erwin, 2000). 

At Fort Knox, home of the U.S. Army’s Armor School, COTS gaming solutions 
have been used to train future tank leaders in order to save money, and most importantly, 
time. Major Michael Muller, USMC, an instructor at the Army’s Armor school and 
Cavalry officer, used the COTS PC-based game TacOpsCav to train everything from 
standard operating procedures to maneuver battle drills (Peck, 2003). Although the 
Armor School (a subordinate of the Army’s Training and Doctrine Command) endorses 
the use of the DoD level simulation JANUS, Major Muller prefers the COTS game 
TacOpsCav™ because it is easier to resource and the train-up time for the interface is 
much less than with JANUS. “I can teach someone to be user-capable with TacOps in a 
half hour. It takes them a day to become a talented user, and they like the game so much 
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that they take it home.” (Peck, 2003). Major Muller contrasts this training time with 
JANUS which requires a three-day block of instruction. TacOpsCav™ saves resource 
planning time in addition to student training time, according to Major Muller. Scenario 
creation for JANUS takes the contractors at the Simulation Center weeks to develop and 
schedule for training officers like Major Muller. Additionally, according to the National 
Simulation Center at Fort Leavenworth, Kansas, JANUS’ installation operational costs 
are two million dollars. With TacOpsCav™, Muller contends that trainers can create 
scenarios within hours and run exercises on any available computers. Although Major 
Muller concedes that JANUS is more realistic that TacOpsCav™, the time that he saves 
using a COTS game is worth more than the increased level of detail JANUS provides. 
“It’s ultimately not as realistic as JANUS, but what does it cost to run TacOps? Nothing” 
(Peck, 2003). 

The U.S. Army Command and General Staff College (CGSC), at Fort 
Leavenworth, Kansas, has also used TacOpsCav™ as a simulation to save time and 
increase training for majors taking brigade staff training courses (Peck, 2004). 
Comparing the use of DoD level simulations CBS (Corps Battle Simulation) and JCATS 
(Joint Conflict and Tactical Simulation) with TacOpsCav™, the cost savings of 
supporting an exercise using the COTS alternative was significant. In the CBS exercise, 
400-500 personnel are required to run a division staff training exercise. The problem, 
notes Jeff Laser, head of the CGSC Digital Leader Development Center, is that only a 
few participants actually train using CBS, and the remaining personnel are required to 
play the opposing force, among other roles, because the artificial intelligence (AI) of the 
CBS is minimal. For the JCATS exercise, 40 students performed duties as a brigade staff 
while an additional 40 support personnel (including retired military officers contracted to 
support the training event) were required to control both friendly subordinate and enemy 
units. In contrast, the trainers using TacOpsCav™ were able to train 600 students with a 
requirement of only 100 support personnel. With TacOpsCav™, the AI allows users to 
simply issue orders and allow the game to do the rest (Peck, 2004). In addition to the 
support cost savings demonstrated by using TacOpsCav™ for the brigade staff training 
event, the COTS game platform has the advantage of brevity according to Lieutenant 
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Colonel Blaise Cornell-d’Echert, tactics instructor and exercise coordinator for CGSC’s 
Advanced Operations Warfighting Course (2004). 

(With JCATS, students) would be able to spend one day executing a plan 
and maybe there would be enough time to run a second time. Because 
(TacOpsCav™) scenarios are shorter, you can do one, have an after-action 
report and do it again with a modified version of the scenario. 

While COTS gaming software has great potential as a military training simulation 
to reduce training costs by saving money, reducing investment risk, limiting resource 
requirements and increasing training through-put, COTS games used as simulation tools 
must be utilized correctly, as these products are first and foremost designed as an 
entertainment product for the commercial marketplace. 

5. Concerns for COTS Use 

Opponents of large scale simulations correctly point out that looking at price tags 
alone can be misleading (Peck, 2003). One of the advantages of developing COTS 
gaming software is that there are no requirements for accurate, real world modeling. 
DoD simulations, alternatively, require very detailed algorithms that are validated using 
real weapons and operational data. For example, high-end simulations may determine 
that a probability of kill (PK) is 90% based on all available real-world data. A 
commercial game company, who is not restricted to real world data, may use a simpler 
algorithm which places a PK at 60% (Peck, 2003). If a simulation is being used to make 
billion dollar force assessment decisions, opponents to using COTS software contend, 
“there is a big difference between a 60% (solution) and a 90% probability of something 
happening” (Peck, 2003). 

“Video games are video games” concludes John Lenyo, vice president for 
business development at BAE Systems Flight Simulation. In an interview in National 
Defense magazine, Mr. Eenyo outlines the differences between a PC game and full 
simulations (Erwin, 2000). 

The reason you can do a game on a PC but you need a full fledged 
simulator to do a simulation is the computing power isn’t there, and there 
are shortcuts everywhere to make it fit on a PC. Huge shortcuts (a $29 
game) will never be like a $29M simulator. The reason it costs $29M is 
because it is much more sophisticated. 
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Mr. Lenyo’s concern is a significant one, and the degree of realism required by 
the simulation must be determined by the trainer before selection of any platform. Mark 
Herman, president of Booz Allen Hamilton and Breakaway Games, concedes this 
argument, but points out that if total realism is not a requirement, COTS games may 
provide a cost effective alternative (Peck. 2003). 

If you want operational code for a weapons system, you wouldn’t use 
commercial game designers. They have a very rapid build process, and 
it’s often a little shoddier. But if it’s for things that don’t get anyone 
killed, you can get a product that is much cheaper and frequently better. 

Ultimately, the level of detail and required use of the simulation must be 
determined by the trainer. Trade-offs must be made in determining if the training value 
of the event is to draw conclusions from the simulation or have the simulation provide an 
environment where teams are learning cognitive tasks not based on conclusion from the 
simulation. 

Adapting COTS gaming software for military training also requires users to place 
control measures in place in order to reduce horseplay, and increase user acceptance of 
the game as a legitimate training venue. Major Chris Sharp, assigned to the College of 
Continuing Education at the Marine Corps University, notes that elements such as 
establishing a chain of command are some of the unique requirements of COTS games 
used by military trainees (Baker, 2001). 

Although the literature identifies areas for concern when using COTS software, 
by defining the intended use of the platform, understanding what conclusions can and 
cannot be provided by a commercial game, and placing control measures for COTS 
application users can leverage this accessible product to meet specific training needs. 

E. CONCLUSIONS FROM LITERATURE REVIEW 

Overall, the literature hints at the possibility of increasing team performance 
through the use of simulations, but does indicate that additional work is needed to capture 
the impact and measurable differences that simulations can provide. Based on what we 
found during the review of the existing literature, we were able to refine our research 
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approach, determine expected findings, and begin to determine what training events 
would be suitable to train using a COTS game. 

Additionally, the debate over the effectiveness of games versus simulators 
demonstrates that additional studies to measure the differences in the training value 
provided by each are required. These studies must determine exactly what tasks a game 
or simulator can and can not be used to train. The literature clearly shows that when a 
high degree of detailed accuracy is required, a true simulator is likely the right tool for 
the job. But when the degree of accuracy is less important than a system’s use to 
facilitate training, a COTS solution may suffice. 
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III. SELECTION OF PARTICIPANTS, SOFTWARE, AND 

METRICS 

A. INTRODUCTION 

The key considerations in conducting our work were the selection of our 
participants, the tasks and/or battle drills we would attempt to train, and the software we 
would use to conduct that training. Upon selecting our participants, we reviewed 
manuals and training references (FM 7-8, Infantry Rifle Platoon and Squad and ARTEP 
7-8-MTP, Mission Training Plan for the Infantry Rifle Platoon and Squad) relevant to 
squad-level tasks. At this time we also played-tested several commercially available, 
multiplayer computer games to evaluate their strengths and weaknesses. 

B. PARTICIPANT SELECTION 

From the beginning, we knew wanted our experiment participants to match our 
target audience: US Army Soldiers training on dismounted squad tactics. Possibilities 
included: junior enlisted infantry Soldiers during their Infantry Advanced Individual 
Training (AIT), junior officers in the Infantry Officer Basic Course (lOBC), or Reserve 
Officer Training Corps (ROTC) cadets. We ultimately selected the junior officers (newly 
commissioned second lieutenants) attending lOBC for several reasons: they were 
scheduled to conduct squad-level training on the tasks we planned to research, our 
familiarity with their training syllabus and the leadership at lOBC, and we wanted an 
audience that we theorized would be familiar with using a computer, as lOBC students 
are required to have some number of college credits if not a completed degree, and we 
assumed these second lieutenants would have some basic knowledge on operating a 
personal computer from their college experience. 

C TASK SELECTION 

To focus to our research, it was necessary to identify a task or drill to train and 
observe in the virtual environment and in the field. Our critical resource for task 
selection was FM 7-8, Infantry Rifle Platoon and Squad, March 2001, which provides 
“doctrine, tactics, techniques and procedures on how infantry rifle platoons and squads 
fight” (FM 7-8, 2001). This reference serves as the basis for training infantry Soldiers to 

conduct collective (squad and higher level) tasks. Fundamental to the execution of 
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collective tasks is the battle drill, which is “a collective action rapidly executed without 
applying a deliberate decision-making process” (FM 7-8, 2001). Battle drills are also 
described as “how platoons and squads apply fire and maneuver to commonly 
encountered situations. They require leaders to make decisions rapidly and to issue brief 
oral orders quickly” (FM 7-8, 2001). ARTEP 7-8 Drill, June 2002, identifies the 
following battle drills: 

• Break Contact (Dismounted) 

• React to Ambush 

• Knock Out Bunkers (Platoon & Squad) 

• Enter and Clear a Building (Platoon) 

• Enter and Clear a Trench (Platoon & Squad) 

• Conduct Initial Breach of a Mined Wire Obstacle (Platoon) 

• Enter Building/Clear Room (Squad) 

• React to Contact (Platoon & Squad) 

• React to a Chemical Attack (Platoon & Squad) 

• React to Nuclear Attack (Platoon & Squad) 

• React to Indirect Eire (Squad) 


With these drills as reference, we began the process of identifying the drill we 
wanted to incorporate into our training transfer research. As a training device, a standard 
computer monitor with typical stereo speakers or headphones severely limits the natural 
cues and inputs a Soldier receives in order to conduct the actions required to support the 
aforementioned battle drills. Considering this inaccurate representation of field of view 
and audio cues, we identified criteria we felt were critical in selecting a battle drill: 

• Reproducible in the virtual environment. This was a screening criterion; if 
it wasn’t possible then the task was not considered. 

• Capable of supporting multiplayer missions with up to thirty players on a 
local area network (EAN). 

• Accurately modeled scenarios and equipment. The software does not 
require an accurate ballistics model. 
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• Peripheral (visual and aural) cues are important, but not critical to 
performance. While peripheral cues are critically important in both the real and 
virtual world, the reduction in cues inherent in using a computer monitor and 
speaker system severely limits the amount of information available. 

• Minimal requirement for precise body awareness. Playing from a first- 
person perspective typically shows only the hands and weapon of the in-game 
avatar, leaving the remainder of the body un-drawn except in a third-person 
perspective. 

While all of the tasks met the first criterion, reproducibility in the virtual 
environment, all but “React to Contact” battle drill are heavily reliant on peripheral cues 
and precise awareness of the avatar’s position in the environment. While peripheral 
cueing and body awareness are necessary in the “React to Contact” battle drill, the fact 
that the attacker has the initiative to decide where and when to attack mitigates these 
factors. Additionally, this battle drill is the fundamental building block of infantry 
maneuvers, and the other drills are extensions of the tasks defined by “React to Contact”. 
In selecting this battle drill, we theorized that if we could determine if the COTS software 
could support the basic building block of infantry maneuvers, then subsequent research 
using COTS software for other battle drills was viable. 

D. SOFTWARE SELECTION 
1. Introduction 

“It’s not teaching how to shoot or to drive; it’s teaching the cognitive” is how 
Ralph Chatham, the DARWARS Program manager, describes what leaders can learn 
from multiplayer PC-based training systems (Walker, 2005). While an entertaining 
process, the task of selecting the right commercial gaming software was challenging. 
Foremost, in keeping with our goal of a less-expensive training device, we wanted a 
game that could be used on systems readily available to Soldiers and units. For this 
reason, we eliminated console systems like the Microsoft Xbox® and Sony Playstation® 
from consideration and focused us on Personal Computer (PC) based software due to 
accessibility of PCs from unit level to computer labs common at most major military 
installations. There was no shortage of titles available that met our screening criteria of a 
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PC-based first-person perspective shooting game that was multiplayer-capable. The 
following paragraphs discuss the criteria we established for selecting software and a 
comparison of the other titles considered but not selected. 

2. Selection Criteria 

Fundamental to our research was finding software that could provide appropriate 
cues that would lead the squad to make tactical decisions without a prohibitive set of 
system requirements. Essentially we wanted to use software that would work on 
computers that our experience has shown to be available to most Army units: a computer 
that is two to three years old equipped with integrated graphics and sound. To meet this 
need, we looked at software that was one to two years old. We acknowledge that the 
newest software titles would likely have the best graphics and sound, but a game that was 
even two years old would likely have graphics and sound that were passable and not so 
bad that they would be distracting from the trainee’s experience. 

We wanted a game that could do the job but not require a cutting edge equipment 
to use. Our criteria were as follows: 

• a game platform that did not require high overall system requirements 

• easy multiplayer set-up and play on a local area network 

• weapons, equipment and missions consistent with current US armaments 

• the ability to easily modify the game space to a viable training venue. 

We needed the flexibility to take existing games and create new mission areas that 
include appropriate landscaping, structures, equipment, and characters managed by the 
game’s artificial intelligence to support the training requirements set forth by the unit 
commander. Because of this, the ease of the tool to modify the game environment was a 
key factor in the game selection. The level editing tool must be intuitive, quickly 
mastered and robust enough to create a believable virtual training venue. 

The following tables show our screening criteria for game selection and system 
requirement evaluation. Table 1 details the commercial software titles we considered and 
their relative strengths and weaknesses according to our criteria. Table 2 describes the 
method we chose for determining computer system requirements. The degree of 


38 



difficulty in creating a level with the game’s level (or mission) editing software is based 
on our assessment of the Graphical User Interface (GUI), numbers of models supported 
and required understanding of computer code in order to script events. 


Table 1. Commercial Game Software Comparison 



PC- 

Based 

Multiplayer 
(>30 goal) 

System 

Requirements 

Modern 
Scenarios / 
Weapons 

Modifiable & 
Degree of 
Difficulty 

America's Army™ 

Yes 

Yes 

(32 Players) 

High 

Yes 

No 

Delta Force 2 ™ 

Yes 

Yes 

(SO Players) 

Low 

Yes 

Yes 

(Easy) 

Delta Force: Black Hawk Down - Team 
Sabre^M 

Yes 

Yes 

(SO Players) 

Medium 

Yes 

Yes 

(Easy) 

Unreal Tournament 2004^“ 

Yes 

Yes 

(32 Players) 

High 

No 

Yes 

(Difficult) 

Joint Operations: Typhoon Rising^“ 

Yes 

Yes 

(150 Players) 

High 

Yes 

Yes 

(Moderate) 

Rainbow Six^“ 

Yes 

Yes 

(16 Players) 

Low 

Yes 

Yes 

(Moderate) 

Operation FlashpoinU“ 

Yes 

Yes 

Medium 

Yes 

Yes 

(Moderate) 

Soldier of Fortune™ 

Yes 

Yes 

(32 Players) 

Low 

No 

No 


Table 2. System Requirement Evaluation for Game Selection 



Operating System 

Processor Speed 

RAM 

Video Card 

HD space 

Low 

Windows 98 

Less than 733MHz 

Less than 
128MB 

Less than 
32MB 

Less than 750MB 

Medium 

Windows NT, Me 
or 2000 

733MHz - l.OGHz 

128MB to 
256MB 

32MB to 
128MB 

750MB to 1.4GB 

High 

Windows XP 

Greater than 1.0 GHz 

Greater than 
256MB 

Greater than 
128MB 

Greater than 1.4GB 


E. SELECTION OF SOFTWARE TRAINING METROLOGY 

I. Introduction 

Our early pilot studies were instrumental in fine-tuning our approach to training. 
The challenge of taking traditionally entertainment-focused software and converting it to 
a training tool was brought to the fore during our early pilot studies. How they were 
trained to use the software, what scenarios were best suited to the task, the composition 
of the opposing forces and the equipment available in the environment were areas we had 
to address to support the transfer of classroom computer executed training to a field 
environment. 
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2. Participant Training 

Our experiment was scheduled to take place while our participants were in their 
7* and 8* week of Infantry Officer training. Prior to our experiment, the lieutenants had 
received training on a numerous individual infantry skills including: prepare and issue an 
operations order, engage a target with individual weapon, and provide a situation report 
to a higher element. Without this base of knowledge on individual tasks, our participants 
may have gained some familiarity with tactical operations from their training in the 
virtual environment, but their ability to transfer those skills to the field environment 
would likely have been reduced by their need to later learn their individual .skills. 

To make the training transition from classroom to the field environment, the 
participants in our study needed to learn how to use the keyboard and mouse control 
scheme. In early pilot studies, we provided a classroom presentation on how to control 
their avatar and how to interpret information the software provides them. After this 
briefing, the participants played one or more of the game’s single player missions for 20 
minutes while the researchers provided assistance and game-play tips. To further aid the 
participants, each workstation had a keyboard and mouse reference card available 
throughout the training. We initially felt the training we provided and their exposure to 
the game environment would be sufficient. Survey results from these early pilot studies 
indicated that the participants felt that there was too much lecture and not enough hands- 
on training. Additionally they felt that the single-player gaming session did little to 
prepare them to work together as a squad. 

Based on this feedback, we altered our training for the remaining two pilot 
studies. Our familiarization and training program evolved to a much shorter classroom 
presentation followed by a group, hands-on training session in the virtual environment. 
To better prepare them to work as a team; we executed game familiarization and training 
using their doctrinal elements: the buddy team, the fire team and the squad. In these 
elements, they were trained and helped each other learn to control their avatar; operate 
weapons and equipment; and execute team operations. The positive results from the 
training in these pilot studies led us to use this as our training and fa mi liarization 
technique during the experiment conducted at Ft Benning, GA. 
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3. Training Scenarios 

Our goal in creating a training scenario was to create a tool that would advance 
the training of our participants. Given our participants, we designed scenarios consistent 
with training they would receive at the Infantry Officer’s Basic Course. Based on this, 
we prepared two game ‘maps’ with four missions on each map. The intent of these 
missions was to gradually increase the level of difficulty by adding additional enemy 
forces and varying the terrain in the objective area. To support the preparation of the 
lOBC students to conduct the squad live fire event, we contacted the unit commander, 
and developed the scenarios for game play that would compliment the classroom 
instruction the second lieutenants received on the conduct of the live fire event. Specific 
descriptions of the virtual maneuver training lanes are detailed in Chapter IV, Appendices 
A and E. contains the operations orders and maneuver graphics that drove the training 
scenarios. 

4. Opposing Force 

This section will deal with our dilemma of using human-controlled or Artificial 
Intelligence (AI) OPFOR. Initially, we planned to use soldiers not affiliated with our 
participant group to act as OPFOR. The advantage of live OPFOR came in its flexibility 
to react to any given situation. Three to four soldiers would provide us a cadre of 
OPFOR and provide the participants with a realistic enemy. After one play-test and a 
pilot study though, we realized that using human OPFOR would provide several 
challenges: logistically it would require more computers and a separate room to work 
from; their ability to control their avatar would be greater than the participant opponents 
in the second squad’s treatment; people want to win, and if they’re not allowed to win (or 
attempt to win), their skill with the game can make them harder to kill. 

While a cadre of human-controlled OPFOR was undesirable, an all AI OPFOR 
was equally unpalatable: their reactions and responses to events in the game environment 
were based on predetermined trigger points that do not take into account the potential 
actions of the training force. The AI OPFOR were very effective at standing on an 
objective 
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Ultimately, we deeided a mix of these worked best. We plaeed an Al-managed 
threat foree in and around the objeetive areas and used one or two human-eontrolled 
agents to draw attention to taetieal areas that were being poorly exeeuted. Flank seeurity 
is an example where a human agent was most useful. As they neared the objeetive, squad 
members eeased to provide seeurity to the left and right, foeusing their attention 
eompletely in the direction of the objective. We would use a human OPFOR agent on 
their flanks to stress the importance of flank security. 

5. Equipment Options 

While basic tactics could be taught with the right collection of weapons from any 
era, we elected to use weapons and equipment currently in use by the U.S. Army. Delta 
Force: Black Hawk Down - Team Sabre™ allowed us to remove equipment from the 
game that was inconsistent with the basic infantry rifle squad. Basic squad equipment 
types that the COTS game supported included: M4 carbine with Close Combat Optic 
(CCO), M249 Squad Automatic Weapon (M249 SAW), M16A2 rifle with M203 40mm 
grenade launcher (M203), M61 fragmentary grenades, AN-M8 HC White Smoke 
grenades, and binoculars. 

F. TASK SELECTION AND ANALYSIS 

1. Overview 

In order to maximize the training effect of simulations, leaders must first conduct 
a task analysis of the collective event in order to determine those skills which can be 
trained through simulation, and those that cannot. Due to the limitations of simulation, it 
is important to set realistic training expectations for both trainers and trainees. 
Additionally, trainers and leaders must determine the potential for negative training 
transfer from the use of simulations and implement control measures to prevent adverse 
team learning. With these thoughts in mind, we believe that given a detailed task 
analysis and proper training expectations, a COTS game could be used as an effective 
training tool. 

2. Task Analysis for Squad “React to Contact” 

For our experiment, we conducted a thorough task analysis of the Squad “React to 
Contact” battle drill using the Occupational Information Network’s 0*NET Descriptors 

(2005). We first examined the individual and crew supporting tasks of “React to 
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Contact”, and then reviewed each performance measure of the battle drill in order to 
determine an initial list of tasks that could potentially be replicated by the COTS game. 

For the individual and crew skills, we determined that the COTS game could not 
replicate all the sensory inputs required to properly train and execute the steps involved in 
most tasks. It did however, provide a “good enough” replication of completed individual 
tasks to support the higher collective performance measures without forcing the Soldier 
to manually execute each step in the task. For example, the individual task “Load, 
Unload and Reduce Stoppage of an M16A2 Rifle” requires a M16A2 and a great degree 
of haptic feedback in order for a Soldier to execute this task. Haptic feedback such as 
pulling the charging handle, loading a new magazine and tapping the forward assist are 
critical in order to train this task to standard. We found that the game does provide 
reasonable visual cues for users to recognize the completion of the task. 

Since we never intended to use the COTS game to train “Load, Unload and 
Reduce Stoppage of an M16A2 Rifle” or other individual Soldier skills, but as a venue to 
train collective performance measures, the visual cues provide an adequate replication to 
support such collective tasks such as “squad/team leaders locate and engage known or 
suspected enemy positions with well-aimed fire, and pass information to the 
platoon/squad leader.” (ARTEP 7-8 Drill, 2-38). For our experiment with the lOBC 
students at Fort Benning, all the individual and crew collective tasks that supported the 
squad “React to Contact” battle drill were trained prior to our use of the COTS game. 

For the collective tasks we used the 0*NET Descriptors to examine all the human 
abilities required for selected leader performance measures. In our previous example, 
“squad/team leaders locate and engage known or suspected enemy positions with well- 
aimed fire, and pass information to the platoon/squad leader” (ARTEP 7-8 Drill, 2-38), 
we identified human abilities such as Spatial Orientation, Near and Par Vision, Hearing 
Sensitivity, Sound Eocalization and others that were required for the human operator to 
properly perform the task (Pleishman, 1995). We then examined our COTS game 
platform to determine the degree of replication that the simulation could render. By 
mapping the human abilities to the COTS game, we created a roadmap for our survey 
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questions and developed an understanding of what COTS games can offer. Appendix F 
contains our Task Analysis results for “React to Contact.” 

3. Findings from the Task Analysis 

We found that the COTS game could not adequately provide sufficient sensory 
cues to train individual or crew tasks to standard. However, the visual and audio cues 
from the game were good enough for human operators to accomplish a replication of 
most individual supporting tasks required to conduct the battle drill “React to Contact.” 
In areas where the COTS game could not directly provide required sensory cues, we 
flagged the human ability requirement of the task for future observation to determine if 
the shortfall had an adverse effect. Using our example, “squad/team leaders locate and 
engage known or suspected enemy positions with well-aimed fire, and pass information 
to the platoon/squad leader” (ARTEP 7-8 Drill, 2-38), the COTS game did not replicate 
the human ability requirement for Far Vision. The subjects were viewing a computer 
monitor approximately 20 inches from their eyes, not trying to discern enemy locations at 
an actual distance. To capture the impact of this type of shortfall, we designed our 
participant surveys to address any sensory cue problems in the game and interviewed the 
lOBC training officer. We felt these two control measures allowed us to fairly examine 
the COTS game as a potential training tool and identifying areas where additional 
training may be required due to the inability of the game to support required sensory 
cues. 
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IV. CONDUCT OF THE EXPERIMENT 


A. OVERVIEW 

For this research experiment, we wanted the COTS game use to be as transparent 
a training venue as possible for the second lieutenants at lOBC. Using this tenet as a 
cornerstone of our experiment, we modeled the event as close as possible to the field 
training exercises (FTX) the lOBC students conduct in preparation for live fire events. 
During the FTX, the trainers establish an initial chain of command, administer a five 
paragraph operations order (OPORD) with maps and graphics, and brief the OPORD on a 
sand table that represents the objective terrain. Following the OPORD brief, the second 
lieutenants conduct rehearsals, back briefs, and begin movement towards the field 
objective that is populated with a live OPFOR element (which is controlled by the trainer, 
and consists of students and cadre in enemy threat uniforms). Once contact is made with 
this OPFOR element, the students, using Multiple Integrated Laser Engagement System 
(MILES) on their weapons, conduct actions on contact. Eollowing the event, the trainers 
conduct an after action review in which students can identify strengths and weaknesses of 
their unit and use those lessons to improve performance during the next event. In general 
terms, for our experiment we simply replaced the field with a virtual environment created 
using the COTS game Delta Force: Black Hawk Down - Team Sabre™. Eor the subjects 
and trainers to accept this methodology (of using COTS software), we felt that we needed 
to maintain as many traditional training elements as possible. Additionally, we felt that 
maintaining the structure of a traditional training exercise would focus the attention of the 
subjects to the tactical and not the technical. 

B. PILOT TESTING AND PRELIMINARY FINDINGS 

In order to prepare for our experiment at Fort Penning, we conducted extensive 
local pilot studies using Army and Navy students attending the Defense Language 
Institute at the U.S. Army Presidio of Monterey and both military and civilian students at 
NPS. Our preliminary pilot studies provided very worthwhile detailed feedback on the 
experiment set up, conduct of software training and refinement of our data collection 
method. The final two pilot experiments provided us with an opportunity to conduct a 
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full dress rehearsal for our actual experiment with lOBC. For future studies in this 
research area, it is highly recommended that multiple pilot experiments be conducted 
(with participants from different backgrounds) in order to allow researchers to be fully 
prepared for the conduct of the test. 

As previously discussed, we selected the infantry squad “React to Contact” battle 
drill, from the ARTEP 7-8 Drill as the task to be trained in the simulation exercise. 
Utilizing the level (or mission editing) software that came with the Delta Force: Black 
Flawk Down - Team Sabre™, we created a virtual environment equipped with a fixed, AI 
driven opposing force (OPFOR) which would provide the majority of resistance to the 
squad. As discussed previously, we planned to bring a live OPFOR avatar into the virtual 
environment to either reinforce a teaching point or to maintain control of the event. 
During the first pilot studies (I-III), we kept the conduct of the event as previously 
described, but changed the conditions of the workstations. For these pilots, we utilized 
students and faculty (as a 7-9 man squad) in a local, FAN supported, computer lab. 

During the first and second pilots, we wanted to determine how to set up the audio 
and communication methods for the participants to interact. We utilized headphones 
with microphone (similar to those which support the Marine Corps DVTE system) to 
provide audio cues and a method for subjects to communicate. We used an open source 
software tool called TeamSpeakTM that provided voice over IP (VoIP) capability to allow 
for group communication within the game environment. We also reduced the lighting in 
the lab and added background noise (Nature Sounds'^’^ Rainforest Soundtrack CD) played 
on a small radio in the back of the room to increase immersive qualities of the virtual 
venue. We found that the VoIP performed poorly due to the proximity of subject’s 
workstations and an approximate one second delay in audio transmission. The subjects 
overheard commands from their leaders and then heard the VoIP transmission. As a 
result, subjects either removed the headphones all together to talk or ignored verbal 
commands. We also found out that lower lighting made finding game control keys 
difficult, and the background sounds did not contribute to an immersive quality. During 
the third pilot, we used the computer speakers vice headphones, and allowed the subjects 
to talk above the audio cues from individual workstation speakers for communication. 
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We found that the ability of the subjects to overhear other workstations speakers led to 
missed or inaccurate audio cues from the game environment. Additionally, we rehearsed 
timing of administering surveys, key events and game interface training during the first 
three pilots. Key findings from the first three pilots included: 

• Using headphones without microphones allowed users to receive the 
accurate audio cues from their position within the game environment. By 
adjusting individual sound levels, users could hear game audio cues and spoken 
commands from other squad members 

• Using a partition to separate the two squad fire teams reduced audio cue 
mismatch from the two fire team leaders 

• Reducing lighting impaired subject’s (particularly novice computer users) 
ability to find game controls 

• The best technique for training users on the game interface 

• Timings of after action reviews, issuing of OPORDs and administration of 
the subject surveys 

Using the findings from these three preliminary pilot tests, we refined our test 
conditions, and finalized our written products (surveys and OPORDs) in order to prepare 
for our remaining two local experiments. Our goal was to use these remaining studies as 
our full dress rehearsal for the Fort Benning experiment. We framed our expected 
learning outcome for the final two pilot studies to focus on usability of the interface, 
ability of the leaders to control unit small arms fires, squad communication, and 
command and control. For our pilot IV and V participants, we selected two groups of 
nine Soldiers and Sailors attending the Defense Language Institute (DLI) to serve as our 
rifle squad. The following will address the formal method and findings from these two 
studies. 

1. Method 

a. Lab Setup 

We utilized a local computer lab for the testing, and reconfigured the 
workstations to allow the fire team members to be grouped together with the squad leader 
in a central position located to the rear and center of the two teams. To accomplish this. 
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we set up two rows of local area networked connected workstations oriented towards a 
sterilized wall (Figure 2). Each workstation in the first row facing the wall had desktop 
computers with 17” to 21” color monitors, three-button (wheel) mouse and keyboard. 
The second row of workstations (located behind the first and oriented towards the same 
wall) contained power and LAN plugs for participants using laptop computers. Both of 
the laptops had a three-button (wheel) mouse in order to maintain consistency between 
the desktop and laptop workstations. Although the additional two laptop computers were 
available, only one was used. 


Although we were unable to provide desktop workstations for all test 
subjects due to resource limitations, we did not observe notable performance differences 
between the laptop and desktop computer users. Each workstation’s minimum system 
requirements included a Pentium III 733 MHz processor (or higher), a 32 MB video card, 
and a 4X speed CD-ROM. Most of the workstations exceeded these benchmarks. The 
final modification to the test site was a partition wall placed between the Alpha and 
Bravo teams in order to limit the viewing areas of the participants to only their immediate 
team members. The squad leader workstation was located behind the partition to allow 
for viewing of both teams to replicate the command and control position of the leader in a 
tactical field formation. 
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With the site established, we then loaded the software on all the desktop 
workstations, one dedicated server station, and two of our personal laptop computers for 
use by test subjects without a desktop workstation. With the software loaded, we 
assigned each workstation a duty position within the game in accordance with the task 
organization of a rifle infantry squad. Each workstation was then assigned the equipment 
authorized by each infantryman’s duty position. For example, the squad’s automatic 
rifleman workstation character within the game was assigned the M249 Squad Automatic 
Weapon, ammunition basic load and grenades authorized by the U.S. Army Table of 
Equipment and Allowances (TOE) for an automatic rifleman. 

Within the game’s player options, we also assigned each member a duty 
position label for the other team members to view once in the multiplayer session. 
Continuing with our automatic rifleman example, we assigned the workstation “A AR” 
for Alpha Team automatic rifleman. Once the entire team entered the multiplayer 
session, the workstation label would appear as a green text box above the participant’s 
avatar for identification by other team members. We considered, but rejected, leaving the 
identification labels off, as they are not present in the real world. After some discussion 
with subject matter experts (SMEs) with at least ten years experience in dismounted 
operations, we determined that squad (or team) members that have been formed for some 
time would be able to recognize the gait and posture of its team members. Because the 
game’s avatars are essentially the same, we concluded that allowing labels would allow 
leaders to command and control more efficiently, and not create negative training transfer 
situation. 

For the tactical orders portion of the experiment, we created a terrain 
model (sand table) that replicated the immediate terrain around the first mission objective 
site. A topographical (1:50,000 scale) map with tactical graphics was posted adjoining 
the terrain model to allow the experimenter to brief the squad OPORD from the map and 
then discuss actions on the objective using the sand table. For squad and team leader use, 
we created and handed out sector sketches on 8.5 x 11” paper that included graphic 
control measures placed over a picture of the target objective map obtained from the 
game. Using the three products, we overcame the game platform’s lack of ability to 
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generate a Military Grid Reference System topographical map and operational graphics 
which are used by infantrymen when conducting tactical real world operations. 

b. Virtual Environment Design 

For the multiplayer missions, we used the Delta Force: Black Hawk Down 
- Team Sabre™ Mission Editor (MED) to create a virtual environment (VE) for our 
participants to tactical operations. Eor the VE, we chose to loosely base the terrain, 
structures, enemy and environmental conditions on a Southwest Asia terrorist insurgency 
scenario. The VE was five square miles in size and contained terrain features similar to 
those of the topographic map used to brief the squad operations order. The participants 
began at a friendly firebase which contained a helicopter hangar, a tactical movement 
training lane, and a UH-60 Blackhawk helicopter for insertion of the squad into the 
mission zone once team training was complete on the tactical movement lane. The 
mission zone contained a moderately wooded area with rolling hills and a river on which 
the four mission objectives were located. 

The tactical movement training lane was placed at the firebase to allow the 
team members to practice tactical movement techniques once in the VE. The 500 x 200 
meter site contained two lanes, each consisting of two bunkers at a start point, two 
obstacles to be used as cover and concealment when negotiating the lane (a broken wall 
section and a pile of three destroyed cars), and an enemy bunker at the end of the lane for 
the teams to focus direct fires and movement direction. A wooden watchtower was paced 
beside the lane to allow the squad leader to observe the fire teams’ training on the lane, 
and make spot corrections. Two enemy insurgents (AI) were placed in the end bunkers 
with a programmed instruction set to engage the participants negotiating the lane in order 
to provide a look and feel of moving under direct fire to the teams. The AI weapon 
accuracy instruction set was set to zero to ensure that no participant avatar was killed 
during the lane training. 

The next feature of the VE was the use of helicopter insertion. During a 
previous pilot study, we observed that test subject’s attention level increased dramatically 
once they boarded a helicopter for an insertion into the tactical mission. With this in 
mind, we maintained the helicopter insertion portion of the VE for the pilot testing. Once 
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training was complete at the tactical movement training lane, the participants boarded the 
UH-60 Blackhawk and conducted a 2000 meter air movement into the mission zone. 

For the mission objectives, we selected targets and sites that would 
normally be assigned a 9 to 11 man infantry squad. In keeping with the scripted scenario, 
we used infrastructure nodes that insurgents may seize in order to disrupt or terrorize 
local civilian populations (radio towers, oil and water pumping sites, etc). We placed 6 to 
9 enemy insurgents (AI) armed with crew served machine guns, rocket propelled 
grenades and AK-47 assault rifles. We set these models to fight from fixed positions and 
engage once the insurgents established visual contact with the test squad. We set the 
weapon accuracy of the AI agents initially at 60%, and then increased it on subsequent 
lanes as the squad became more proficient in the VE. 

At the end of each objective, we placed health and ammunition game 
icons that permitted each team member to refresh their ammunition basic load and 
avatar’s health prior to continuing to the next mission site. A game spawn point (which 
allows players to return to a specific location in the game if killed) was placed after each 
objective in order to get subjects back into the VE and not have to move 2000m to link 
back up with their unit (in the OPORD, we referred to these as checkpoints, and required 
our subject squad to secure them in the coordinating instructions of paragraph III). These 
features allowed us to use one VE saving game loading time thus increasing our number 
of mission iterations we could execute with the participants. 

Einally, the server workstation was loaded with an open source video 
capture program (Eraps) that allowed us to capture video of the squad within the VE at 
critical points during the mission. We entered the VE from this workstation as an enemy 
player, so as to not appear on the test participant’s Head-Up Display (HUD), move to a 
position that afforded a key vantage point of each objective, and use the Eraps program to 
capture video of the squad’s actions on contact, movement techniques, and fire control 
and distribution. The intent of the captured video was for use during an after action 
review with participants following completion of a mission. Although not used in this 
manner during the pilot studies, we did validate the method of capturing game play for 
AAR purposes. 
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The test participants started the experiment in a classroom where they 
completed the demographics portion of the survey, received an overview of the 
experiment, a block of instruction on the fundamentals of infantry squad movement 
techniques, the squad attack, and game console controls. Once complete, the participants 
moved to the test lab where they were assigned a workstation by rank for leaders, then 
random assignment for the remaining positions, and completed the second portion of the 
interface training. Once complete, we briefed the tactical operations order to all 
participants, handed out the appropriate maps and graphics, and directed the participants 
enter the VE in a multiplayer session. The operations order directed the squad to conduct 
rehearsals on the tactical movement training lane (Figure 3), and once complete, the 
squad would board the UH-60 Blackhawk for air insertion to the first mission location 
(Figure 4). 



Figure 3. In-Game Five Fire Range 
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Figure 4. Helicopter insertion 


Time was afforded the leaders to review the order with the members of the 
squad on the terrain model or maps, and to develop initial standard operating procedures 
(SOP) for movement, communication, and actions on enemy contact. Following each of 
the missions, the test group completed a post game survey to capture learning and skill 
acquisition progression. At the completion of the last mission, the participants completed 
an end-game survey and received a debriefing. 

2. Results from Pilot Studies IV and V 

The participants of pilot IV were U.S. Army Soldiers from the Defense Language 
Institute (DLI) located at the Presidio of Monterey, California. The Soldiers were 
completing foreign language training at DLI in order to serve as ground interpreters for 
deployed maneuver units. The nine member group had conducted some training in 
infantry squad tactics during periodic field exercises, but not as a fixed group. The 
experience base of the grouped ranged from Staff Sergeant (E6) to Private First Class 
(E3). Three of the nine participants were female, two participants were National 
Guardsman, and the ages ranged from 20 to 34. We also gathered general information on 
video game playing experience and rifle hunting background in order to provide a 

broader assessment of the participant’s level of expertise. 
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We had two interesting findings from our pilot IV experiment. The first involved 
an improved participant assessment of the squad’s ability to conduct small arms “Fire 
Control and Distribution”. We used a five point Likert scale for training confidence 
assessment (1 = Very Unsure, 2 = Unsure, 3 = Neutral, 4 = Confident, 5 = Very 
Confident). Six of six participants with less than two years military experience indicated 
that their confidence of their squad’s ability to conduct “Fire Control and Distribution” 
improved after conducting the games-based training. Of the remaining three participants 
with more than two years military experience, none reported a change to their initial 
assessment. This sustained assessment could indicate that the participants with more than 
two years military experience did not feel the training had a negative impact (Table 3). 

Table 3. Pilot Study Fire Control and Distribution (Squad Assessment) 


Fire control and distribution 




y yV . 

Squad’s Training Level to 
Coordinate Direct Fires 
Against an Enemy Position 

■ Final Confidence In The 
Squad’s Training Level to 
Coordinate Direct Fires 
Against an Enemy Position 

y 



-l-^^^ ^^^^^- 1 




Our second finding was a reported improvement for confidence level of 
conducting individual movement as a member of a squad. Four of six participants with 
less than two years military experience assessed an increased confidence level to 
maneuver under fire as a member of a squad. Of the five remaining participants, all 
indicated that the games-based training event either sustained or improved their 
assessment of their individual ability to conduct maneuver under fire as a squad member 
(Table 4). 
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Table 4. Pilot Study Participant Individual Maneuver Confidence 


Individual Maneuver Confidence 



* Initial Confidence Level to 
Maneuver Under Fire as a 
Squad Member 
■ Final Confidence Level to 
Maneuver Under Fire as a 
Squad Member 


The participants from pilot V were 10 U.S. Navy Sailors attending the DLL The 
results of their pilot mirror much of those found during pilot IV, but in the area of initial 
self-assessed confidence level to conduct tactical maneuvers as a member of a rifle 
infantry squad, the Sailors, who never conducted any formal training other than the block 
of instruction we provided, rated themselves as very high. This result gave us cause for 
concern that our Fort Benning subjects would overrate themselves based on either their 
lack of experience or demand characteristics. Because of this finding, we exercised 
increased caution when we analyzed both our pilot and Fort Benning experiments. 

3. Discussion from the Pilot Studies 

One of the senior participants in our pilot studies indicated that at first, his 
frustration was with the interface, but towards the end of the exercise, his frustration was 
with his ability to maneuver his squad. Based on his comments and participant 
comments from previous sessions, we felt that the training event portion of our 
experiment was having an effect on the target audience. Frustration concerning 
command and control is a common experience during field exercises, so hearing his 
comments meant that he accepted the interface, and was focused on the collective task. 
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Possible confounds in our pilot data include the lack of experience base of the 
individual squad members and the fact that our participant squads do not normally train 
together makes team assessment difficult to measure or validate. Additionally, the 
participants from DLI are language speakers just completing their initial Army or Navy 
training, so any experience and recommendations whether to use this type of training 
remained unresolved at the conclusion of the pilot study. 

C. FORT BENNING EXPERIMENT 


Based on the results of the pilot studies, we refined our experiment for the 
students at lOBC. We made adjustments to the survey, and refined the game 
environment to fix game play errors discovered during the pilot testing. The conduct of 
the Fort Benning experiment mirrored that of our previous pilot tests with the subjects 
completing demographic surveys, receiving a blocks of instruction and practical exercise 
on the game controls, and conducting tactical missions in the VE. The Fort Benning 
computer lab we used for our experiment contained an overhead projector and screens 
that we used for the game training and Fraps video playback for the AAR (Figure 5). 
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Figure 5. Fort Benning Test Site Fayout 


Upon arrival at Fort Benning, we linked up with the lOBC chain of command to 


provide them an overview of the experiment and secure the test location. After the 
commander assigned a platoon to support our experiment, we met with the platoon 
training leadership and administered the Demographics portion of the survey to all four 
squads. The lOBC company commander assigned the trainer from the platoon who 


provided the four squads (two test and two control squads) to be present for our 
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experiment in the computer lab. The platoon trainer, an infantry Captain who served as 
the TAG officer for 4* Platoon, helped us immeasurably in control of the overall test 
from ensuring squads reported in a timely manner to providing insight to training updates 
from the Infantry Center. 

Originally scheduled as four four-hour sessions, we conducted the experiment 
with the two lOBC test squads in the computer lab over two-eight hour training sessions 
due to lOBC schedule conflicts. During the lab portion of the experiment, we allowed 
the 4* Platoon TAG to enter the VE (as an OPFOR player) in order to observe his 
squad’s performance. Under our direction, he actively engaged the test squad in order to 
reinforce learning points the lOBC students received during previous classroom 
instruction, such as failure to provide 360 degree security or not massing small arms fires 
on the objective. At the completion of the lab portion of the experiment, the test subjects 
were instructed not to discuss the experiment with their peers in the control squad until 
after the FTX conducted at the Ware live fire range complex the following week. 

We deployed to Ware range, after lOBC arrived at the live fire range, and with 
the 4* Platoon trainers, observed the test and control squads conduct a blank fire exercise 
on a squad “React to Contact” lane in preparation for the live fire exercise later in the 
week. We observed the performance of all four squads from receipt of mission, through 
execution, and to the AAR. After the AAR, we administered the final portion of the 
survey and conducted our debriefing. As the individual test and control group completed 
their survey, we conducted an open forum interview in order to capture anecdotal data 
from their experience. Finally, we conducted an interview with the 4* Platoon TAG 
officer to capture his thoughts and observations from the experiment. 
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V. ANALYSIS OF EXPERIMENT 


A. INTRODUCTION 

The results presented in this chapter are based on survey results, researcher 
observations and interviews with subjects. Throughout our surveys, we found very few 
statistically significant results. We feel that this is largely based on two factors: the small 
number of participants in the research (41) and our participants’ overestimation of their 
abilities. The small number of participants was largely a result of the time available to 
conduct our research - it was the inflation of self-reported data that made our data harder 
to demonstrate change. 

As stated, our survey results were primarily self-reported opinions of the 
participants’ skill and training confidence in various tactical areas. Our final pilot study 
was the first time we recognized that for some survey questions, our scales were not well 
anchored, which led to some participants reporting an ability level that was clearly 
inflated. In the final pilot study, our participants were a group of U.S. Navy Sailors from 
the Defense Language Institute. During the introductory training we provided, they 
advised us that they had no training in field tasks. After our 20 minute class on tactics 
and individual movement techniques, they reported a level of confidence in tactics and 
maneuver that was not consistent with the amount of experience and training they had 
received. While we know the training we provided was doctrinally correct, the 
participants uniformly assessed their skill in tactics as a four or five on a one to five scale 
where higher is better. 

B. EXPERIMENTAL RESULTS 

I. Demographic Survey Results 

Our demographic survey was aimed at determining the baseline of our 
participants and finding any confounding factors within their past experiences. The 
survey covers personal information, military experiences, and commercial game and 
military simulation experience. We did not find any significant differences between the 
participant groups or other confounding factors as a result of our demographic data 
collection. 
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As stated in Chapter IV, we randomly selected two squads to receive the 
treatment with the other two squads acting as the control group for the experiment. Table 
5 shows the age and number of years of military experience broken down by control 
group and treatment group. The soldiers in the platoon are assigned to squads in 
alphabetical order, so we were surprised to see the age difference between the two groups 
(control 28.1 and treatment 25.8). 61% of the participants have prior military experience 
with the rates being similar between the groups (62% with experience in the control 
group and 60% in the treatment group). While over 60% have prior military experience, 
of those, less than 5% have served in a combat zone with a ground unit. 


Table 5. Participant Age and Military Experience Prior to lOBC 



Group 

N 

Mean 

Std. Deviation 

Std. Error 
Mean 

4. Age: 

Control 

21 

28.10 

4.867 

1.062 

Treatment 

20 

25.80 

3.778 

.845 

12. Do you have any prior 
military experience?: 

Control 

21 

.62 

.498 

.109 

Treatment 

20 

.60 

.503 

.112 

16. Have you served in a 
combat zone with a ground 
unit?: 

Control 

12 

.17 

.389 

.112 

Treatment 

13 

.15 

.376 

.104 


Given our research area, we also looked at what percentage of our participants 
had experience playing commercial computer or other types of video games. We looked 
at both the frequency that they currently play commercial games as well as the frequency 
with which they played games in the past two years. We were particularly interested in 
the gaming experience in the treatment group; knowing that their familiarity with the 
game control scheme would affect their ability to perform in the virtual environment. 
That comfort in the virtual environment may indicate a reduction in the amount of 
attention they would require to control their avatar, which would potentially affect their 
ability to execute the required “React to Contact” tasks. 


60 




The results showed that while 75% (31 of 41) of the participants reported playing 
games at least monthly in the past two years, only 44% (18 of 41) report playing games 
monthly or more often. Table 6 shows the results for the control and treatment groups. 
Table 6. Commercial game play experience for the participants by group. 


20. In the past two years, how often did you play 
video games? 



21. How often do you now play video games? 



Given our use of a multiplayer, first person shooter for our study, we surveyed the 

participants for their experience with first person shooter games and multiplayer games. 

Within the treatment group, 60% (12 of 20) had played first-person shooters and 80% (16 

of 20) had played multiplayer games. The results were similar with the control group, 
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with 62% (13 of 21) having some experience with a first-person shooter and 72% (15 of 
21) having experienced a multiplayer game. 


In an effort to identify other potentially confounding factors between the groups, 
we surveyed participants on areas related individual skill in the field environment and 
squad socialization. We found no significant differences between the two groups in these 
areas. In our survey we asked for experience rifle-hunting, with more frequency being 
better, as a basis for individual skill in moving through the field and identifying a target 
in a cluttered, woodland environment. The results were not significantly different 
between the groups (control: 38% yes, treatment: 35% yes). From our experience with 
small unit dynamics, we felt that the amount of time a squad spends together may 
indicate a level of synergy within the unit. Numbers skewed in either the low or high end 
of the scale shown in Table 7 could indicate an additional factor present in the squad’s 
performance. The results found the squad socialization data to be similar between the 
two groups. 


Table 7. Squad socializing results by participant groups. 




Never 

Rarely 

Seldom 

Often 

Daily 

Mean 

(sd) 

30. Do you socialize with 
your squad after duty 
hours?: 

Control 

0% 

4% 

38% 

38% 

9% 

3.33 

(.232) 

0/21 

1/21 

8/21 

8/21 

2/21 

Treatment 

0% 

15% 

25% 

40% 

20% 

3.65 

(.221) 

0/20 

3/20 

5/20 

8/20 

4/20 


2. Post-Mission Survey Results 

Upon the completion of each mission in the virtual training environment, 
participants filled out a post-mission survey covering: their performance in the game 
environment, their duties in the squad, their actions during the mission, and any 
comments or recommendations they had for the training. This survey section was 
completed only by the treatment squads. The squad that underwent the treatment first 
executed seven missions in the virtual environment. Due to time constraints, the second 
squad to undergo the treatment was only able to execute six missions. At the completion 
of the treatment stage of the experiment, we received 132 Post-Simulation Mission 
surveys. The comments section of this survey will be covered in section 3, Simulation 
results. The results in this section will focus first on the participants’ reported 
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performance in the environment and then look at what they perceived to be the training 
value of executing these tasks using a commercial PC game. 

First and foremost, we wanted to determine how the participants felt about their 
performance in the environment. Table 8 shows how the participants reported their 
competence with the game controls and environment on a scale from one to ten. The first 
table shows the mean score by mission number. Table 9 shows the complete information 
in a box plot format. With the exception of mission four, the treatment subjects reported 
improved performance in the environment on a relatively consistent basis. After 
reviewing our notes, we found that mission four occurred immediately after our evening 
break for dinner, which may have been a factor in their scoring and game-play 
competence. 


Table 8. Self-reported competence mean score by mission. 
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Table 9. Self-reported competence box plot. 
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One concern we had with using a computer as a training tool for squad maneuvers 
was the ability of the participants to actually maneuver as a squad. The restricted view 
frustum provided by a traditional computer monitor limits peripheral vision, which is 
used to maintain position in a formation. To assist the player with finding objectives and 
squad members in relation to the player’s position, Delta Force: Black Hawk Down - 
Team Sabre™ provides the player with a circular, overhead moving map in the lower 
right comer of the display. 

This moving map displays an icon for friendly forces and squad objectives and 
can be zoomed closer and further from a position above the player. Figure 6 shows a 
typical display from the player’s perspective. The moving map with a blue diamond icon 
representing the player is the circular area in the lower right-hand comer of the display. 
The moving map technique proved to be suitable to the treatment group over the course 
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of the experiment, with over 88% of the post-mission reports rating their ability to 
maintain position in the formation as “Good” or better. 



Figure 6. Screen shot from Delta Force: Black Hawk Down - Team Sabre™ 

Over the period of time required to conduct the treatment, the performance of the 
older graphics from the game engine and the integrated graphics chip did not negatively 
affect their ability to observe and engage the enemy. 74.2% (98/132) of the post-mission 
surveys indicated that the participants observed the enemy during the mission, while 72% 
(95/132) indicated they engaged the enemy. Of those participants who engaged enemy 
forces, 78% (103/132) reported they were able to observe the affect of their actions. 

In addition to visual factors affecting their performance, we surveyed the 
treatment group on their ability to communicate with members of their squad during each 
mission. Given that the squad used headphones to provide audio cues but had to use 
voice commands in the room to direct and report, our participants were initially 
concerned about their ability to communicate with other squad members. Results from 
the survey and observation of their actions in the training environment showed that 
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ultimately the squad had little trouble communicating (Table 10). In fact, participants 
stated that this exercise “improved our squad’s communication” and “forced the user to 
communicate and organize (information) to standard.” 


Table 10. Squad Communication assessment after each game mission 



Poor 

Fair 

Good 

Very 

Good 

Excellent 

9. Rate your ability to communicate with your superior: 


2.30% 

55.70% 

29.80% 

12.20% 

0/132 

3/132 

73/132 

39/132 

16/132 

10. Rate your ability to communicate with your 

Teammates 


3.10% 

57.30% 

29.80% 

9.90% 

0/132 

4/132 

75/132 

39/132 

13/132 

11. Rate your ability to communicate with your 

Subordinates 


5.30% 

55.30% 

29.80% 

9.60% 

0/132 

6/132 

63/132 

34/132 

11/132 


3. Post-Simulation Exercise Results 

Once the complete series of missions in the virtual training environment was 
executed, the treatment group completed a survey assessing thoughts on the experiment 
as a training event, their confidence in executing tasks as an individual and as a squad, 
the effectiveness of training using COTS software, and any additional comments they had 
on the event. This survey after training in the virtual environment was the final step in 
our treatment prior to the platoon deploying to the field for their squad-level training 
exercises. 

One of the key questions we wanted to answer in this section of the experiment 
was whether Soldiers would accept an entertainment-focused COTS product as a training 
device. Although we made no effort to disguise the training tool - it was clearly marked 
as a commercial computer game product - only 19% (4/21) stated they felt like they were 
playing a game during the study. The results listed in Table 11 demonstrated that the 
overwhelming majority of participants felt that they were conducting training (81% 
selecting this statement as “True” or “Very True”). It was also found that participants did 
not believe their actions in the game environment were without consequence (66.7% 
believed a statement “that their actions had no consequences” was “Not True” or 
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“Somewhat True,” with none of the group identifying that statement as “True” or “Very 
True”). On the overall experience of using a COTS game as a training tool, one 
participant stated: 

I was skeptical (about the game) at first, but (I was) converted by the end. 
Without a facilitator leading AARs and pushing the squad to improve 
upon weaknesses, this would not have been worthwhile. 


Table 11. The COTS Virtual 1 

fraining Environment as a training tool 


Not True 

Somewhat 

True 

Neutral 

True 

Very 

True 

26. Rate this statement's description of your 

experience: "During this exercise, Ifelt like my 
actions in the virtual environment had no 
consequences. ” 

42.90% 

23.80% 

33.30% 

- 

- 

9/21 

5/21 

7/21 

0/21 

0/21 

27. Rate this statement's description of your 

experience: "During this exercise, Ifelt like I was 
playing a game." 

19% 

57.10% 

4.80% 

19.00% 

- 

4/21 

12/21 

1/21 

4/21 

0/21 

28. Rate this statement's description of your 

experience: "During this exercise, Ifelt like I was 
conducting training." 

- 

14.30% 

4.80% 

42.90% 

38.10% 

0/21 

3/21 

1/21 

9/21 

8/21 


While the participants clearly indicated they believed that COTS games could be 
used to conduct training, it was important that they were aware of the skills that the event 
was intended to train. Our focus was on training the squad “Attack” and “React to 
Contact” battle drills, and from the participant’s feedback, the training provided left the 
students feeling that the experience was good training for themselves as individuals and 
the squad as a whole. Table 12 contains the detail on participant responses with all 21 
respondents rating the training as “Good” or better in both individual and squad 
categories. 


Table 12. Training effectiveness on the Squad “Attack”/ “React to Contact” battle drills 



Poor 

Fair 

Good 

Well 

Very 

Well 

23. Rate the simulation exercise’s effectiveness to train 
you to conduct a squad “Attack”? 

- 

- 

38.10% 

42.90% 

19.00% 

0/21 

0/21 

8/21 

9/21 

4/21 

24. Rate the simulation exercise’s effectiveness to train 
your squad to conduct a squad “Attack”? 

- 

- 

33.30% 

52.40% 

14.30% 

0/21 

0/21 

7/21 

11/21 

3/21 
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When we identified the platform and software for our study, a key concern that 
arose from using older software was “would the audio and visual cues allow the 
participants recognize the threat force when it arose?” Older graphics and the reduced 
capabilities of an integrated graphics and audio on our experiment computers would 
produce less robust cues than software and hardware built to support computer gaming, 
and significantly less cues than the real field environment. And while the treatment 
group felt that their training with the virtual environment was effective, they did have less 
favorable opinions about the audio and visual cues presented by our training tool (see 
Table 13). 


Table 13. Audio and Visual Cues from the virtual training environment. 



Poor 

Fair 

Good 

Well 

Very 

Well 

19. Rate the simulation's ability to provide anrfio 
cues that allowed you to identify the location 
of enemy fires?: 

9.50% 

28.60% 

23.60% 

28.60% 

9.50% 

2/21 

6/21 

5/21 

6/21 

2/21 

21. Rate the simulation's ability to provide anrfio 
cues that allowed you to determine the 
location of vour squad members?: 

19% 

42.90% 

14.30% 

14.30% 

9.50% 

4/21 

9/21 

3/21 

3/21 

2/21 

20. Rate the simulation's ability to provide visual 
cues that allowed you to identify the location 
of enemy fires?: 

- 

4.80% 

33.30% 

33.30% 

28.60% 

0/21 

1/21 

7/21 

6/21 

6/21 

22. Rate the simulation's ability to provide visual 
cues that allowed you to determine the 
location of vour squad members?: 

- 

9.50% 

28.60% 

38.10% 

23.80% 

0/21 

2/21 

6/21 

8/21 

5/21 


The most frequent issue related to audio and visual cues was the limitations on 
peripheral vision. Comment sections on the survey include requests for “better peripheral 
vision” and “more peripheral feedback.” In addition to comments on peripheral vision, 
participants requested the ability to “see where the squad leader and team leader were 
pointing” in the environment. In a tactical situation, it is common for leaders to use hand 
and arm signals to direct their Soldiers and identify targets or fields of fire by pointing. 
Some leaders solved this by getting up from their computer, walking to a squad 
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member’s station and pointing out an objeet on the eomputer display. While this is an 
effeetive technique for providing guidance, it leaves that leader’s avatar defenseless in 
the virtual environment. 

One theme that showed up typically in early Post-Mission surveys was that the 
participant “didn’t know what to look for” in the environment. This comment or a 
similar comment on being challenged to identify the enemy was made by four members 
of the treatment group within their first three missions. The comment was likely a factor 
of their unfamiliarity with the game environment as it was not repeated in their later Post- 
Mission surveys. 

Closely tied in with audio and visual cues was the player’s ability to make sense 
of the data presented and gain situational awareness. The majority of participants, 85% 
(18/21) reported that the software provided them “Good” or better feel for the situation 
during the virtual training exercise. Of the remainder, only one participant reported the 
situational awareness from the exercise to be “Poor,” the worst rating available. Table 
14 details the results from this question. 


Table 14. Participant’s measure of the situational 

1 awareness. 


Poor 

Fair 

Good 

Well 

Very 

Well 

25. Rate the simulation exercise's effectiveness to provide 
you with situational awareness? (How I see mvself & the 
enemy) 

4.80% 

9.50% 

23.80% 

42.90% 

19.00% 

1/21 

2/21 

5/21 

9/21 

4/21 


Given the above factors, what is the impact on the confidence of our participants? 
Did this event bolster their confidence in the abilities they have learned in lOBC or did it 
open their eyes to the many elements of the task that a classroom session does not prepare 
them for? One participant stated “.. .1 could tell that during the course of the training my 
squad was becoming more proficient.” The survey results supported that view, showing 
a slight improvement in the treatment squads’ confidence on maneuvering under enemy 
fire as a member of a squad, from a mean of 4.07 to a mean of 4.19. A similar result was 
obtained when comparing the treatment group’s initial responses on the Demographic 
survey and the Post-Simulation Survey. Confidence in the squad’s training level to 
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coordinate direct fires against an enemy position rose from mean 3.76 to 3.95 while 
confidence to maneuver under enemy fire rose from 3.63 to 3.95. While we were 
encouraged by the increased confidence our participants reported, only the results from 
the field would demonstrate that the skills learned in the VE could be transferred to real 
world actions. 

4. Field Training Results 

As previously discussed in chapter IV, the post-FTX survey was conducted on the 
test and control squads as they completed the blank fire exercise on the Ware range 
complex. We used two separate post-FTX surveys (one for the test and one for the 
control squads), and conducted an open forum interview with each squad after 
completion of the survey and a one-on-one interview with the 4* Platoon TAG officer. 
Findings from the survey data overall showed no significant difference between the test 
and control groups on the self-assessed questions concerning training confidence on 
individual movements, fire control and communications within the squad. However, on 
the question of how well the squad conducted actions on contact (using the blank fire 
exercise as the benchmark), the platoon TAG assessed the test group’s performance 
higher than the control group. 

Of the question “how well did your squad conduct actions on contact?” with a 
Fikert 5 point scale (1 = Poor, 2 = Fair, 3 = Good, 4 = Well, 5 = Very Well), the test 
groups assessed themselves a mean of 4 (Well), and the control groups rated themselves 
slightly higher than good (3.57) (Table 15). Using a one-way ANOVA, we were able to 
determine a 0.12 level of significance on the question of how well the subjects assessed 
their squad’s ability to conduct actions on contact (Table 16). 


Table 15. Assessment on conduct of actions on contact (Five) 






95% Confidence Interval for Mean 


N 

Mean 

Std. Deviation 

Lower Bound 

Upper Bound 

Control 

21 

3.57 

.811 

3.20 

3.94 

Test 

20 

4.00 

.918 

3.57 

4.43 
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Table 16. Significance of assessment of conduct of actions on contact (Live) 



Sum of Squares 

df 

Mean Square 

F 

Sig. 

31. How well did 
your squad 
conduct actions 
on contact? 

Between Groups 

1.882 

1 

1.882 

2.518 

.121 

Within Groups 

29.143 

39 

.747 




Despite the fact that the number of subjects is low and the data comes from a self- 
reported assessment of a group of relatively inexperienced subjects (initial entry training 
lOBC second lieutenants), the fact that they felt their squad did well indicates a degree of 
confidence in the group’s ability to conduct this complex task - whether based on the 
additional training they received or not. When conducting ground operations at the small 
unit level, where each doorway and alley pose a lethal threat, confidence in your unit 
(and fellow infantryman) is essential for effective operations. Whether real or perceived, 
infantry leaders at the squad level need for their Soldiers and Marines to trust in their 
training, because hesitation by any man can have catastrophic results. This data point, 
regardless of the confounding variables, is important because it demonstrates this 
confidence. 

The treatment group’s post-FTX survey also included eleven questions that 
elicited an assessment of the simulation (or game) exercises on their squad training level. 
The data points to areas in which games utilized in a manner similar to the experiment 
may be a viable training alternative and other areas where they may not. First, we asked 
the subjects to assess what impact the simulation exercise had on their individual 
performance, then their squad performance. We found that none of the 22 respondents 
reported any negative impact on their training, with the specific assessment rating 
varying based on the task at hand. 

Of the overall assessment of the simulation exercise on their individual 
performance, all respondents stated that it had a positive effect on their performance, with 
81.8% reporting it as “Very Positive” and 18.2% reporting it as having a “Positive” effect 
(Table 17). 
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Table 17. Participant assessment of the simulation on performance 



Very 

Positive 

Positive 

No 

Effect 

Nesative 

Very 

Negative 

38. What impact did the simulation exercise have 
on your performance? 

81.80% 

18.20% 

- 

- 

- 


When it came to cognitive tasks such as assessing the situation or understanding 
of squad maneuvers, we found that the experience in the virtual environment rated high, 
but for tasks that require a type of tactile response, the results were mixed. For example, 
when asked about the impact of the simulation exercise on understanding squad 
maneuvers, responses were 68.2% positive and 31.8% very positive, but when addressing 
ability to use available cover or concealment for maneuvering against an enemy, 40.9% 
of respondents indicated the exercise had no effect in their ability to use cover and 36.4% 
responded no effect for concealment (Tables 18 - 19). This may be due in part to the 
limited field of view that the COTS game can provide versus the full view in the real 
world and the tactile feedback required by the tasks of finding cover and concealment. 


Table 18. Simulation impact on individual understanding of squad maneuvers 



Very 

Positive 

Positive 

No 

Effect 

Negative 

Very 

Negative 

47. What impact did the simulation exercise have 
on your understanding of squad maneuver? 

31.80% 

68.20% 





Table 19. Simulation impact on individual training of cover and concealment use 



Very 

Positive 

Positive 

No 

Effect 

Negative 

Very 

Negative 

44. What impact did the simulation exercise have 
on your ability use available cover to maneuver 
against the enemy? 

18.20% 

40.90% 

40.90% 



45. What impact did the simulation exercise have 
on your ability use available concealment to 
maneuver against the enemy? 

18.20% 

45.50% 

36.40% 
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Additional results from the survey indicate that the exercise was either positive of 
had no effect on the subject’s ability to assess the situation when in contact and maneuver 
as a squad member (Table 20). For individual ability to scan, detect and identify the 
enemy, 22.7% of respondents indicated no effect on this task from the exercise (Table 
21). This may be due in part to the limitation of the COTS game platform to render 
distant objects well or the lack of “visual noise” within the game environment. We use 
the term “visual noise” to refer to additional objects such as birds, dust, blowing grass, 
etc. Visual static is an element of the real world that is important in training the task of 
scanning and detection of enemy forces. The ability to filter this static requires training 
in the presence of this clutter, something that none of the COTS games we considered 
could provide. However, many of the current state of the art games are beginning to 
provide this visual noise. 


Table 20. Simulation impact on training 

assessing 

situation and maneuvering und 

er fire 


Verv 

Positive 

Positive 

No 

Effect 

Negative 

Verv 

Negative 

39. What impact did the simulation exercise have 
on your ability to assess the situation when in 
contact? 

18.20% 

72.70% 

9.10% 



40. What impact did the simulation exercise have 
on your ability maneuver under enemy fire as a 
member of a squad? 

36.40% 

54.50% 

9.10% 




Table 21. Simulation impact on individual ability to detect enemy 



Verv 

Positive 

Positive 

No 

Effect 

Negative 

Verv 

Negative 

46. What impact did the simulation exercise have 
on your ability scan, detect and identify the 
enemy? 

27.30% 

50.00% 

22.70% 




The subjects’ assessment of the impact of the simulation on their squad’s abilities 
was mixed. In the assessment of the impact on the squad’s overall performance, 
communication and ability to maneuver under fire, over 80% of the subjects reported a 
“Positive” or “Very Positive” impact from the exercise with the remainder reporting it 
had no effect (Table 22). 
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Table 22. Simulation impact on squad performance, communication and maneuver 



Verv 

Positive 

Positive 

No 

Effect 

Negative 

Verv 

Negative 

48. What impact did the simulation exercise have 
on your squad’s performance? 

27.30% 

54.50% 

18.20% 



49. What impact did the simulation exercise have 
on your squad’s ability to communicate 
information? 

36.40% 

50.00% 

13.60% 



51. What impact did the simulation exercise have 
on your squad’s maneuver under enemy fire? 

27.30% 

54.50% 

18.20% 




In the area of the squad’s ability to control direct fires, while over 70% reported a 
“Positive” or “Very Positive” effect, 27.3% of the subjects indicated that the simulation 
had no effect (Table 23). This response was higher than anticipated, as earlier pilot study 
results rated the impact of the game on the control of direct fires higher than that reported 
by the lOBC students. This may be due to the direct fire control methods taught by 
lOBC which include leaders physically touching operators of key weapon systems to 
indicate direction of fire, and the use of the AN/PEQ-2 infrared illuminator aiming 
designator for directing fires during night operations. Both of these methods could not be 
accurately represented by the COTS game exercise. 


Table 23. Simulation impact on squad’s ability to control direct fires 



Verv 

Positive 

Positive 

No 

Effect 

Negative 

Verv 

Negative 

50. What impact did the simulation exercise have 
on your squad’s control direct fires? 

27.30% 

45.50% 

27.30% 




Our most interesting data came from the interviews conducted with the 4* Platoon 
TAG officer and test subjects in the open forum discussion following the post-FTX 
survey. The TAG officer indicated that he observed a moderate difference between the 
test and control squads in overall performance. These differences included the ease at 
which the test squads were able to get into movement tactical formations (traveling, 
traveling overwatch and bounding overwatch) and generally being more aware of their 
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surroundings (in the field site). Additionally, the test groups communicated spacing and 
distance better at the assault position than the control groups. 

The TAG noted that the test squad’s planning for the live event was better than 
the control squad’s. Specifically, he observed that the test squads used binoculars more 
often at security halts during the FTX and some of the test squad members actually 
purchased civilian binoculars in preparation FTX. He observed, as did we, that the test 
groups utilized the binoculars often during the game play to increase their ability to detect 
enemy forces in the virtual environment, and that the use of binoculars transferred 
directly from the game experience to the FTX for the test squads. In addition to 
binocular use, he observed a direct fire planning difference between groups. He stated 
that the test squads discussed the use of visual signals more for fire control, and assigning 
specific tasks for key weapon systems on the objective such as using the M203’s 40mm 
grenade for clearing defilades. We asked the TAG if he received any feedback from the 
test groups with regards to the GOTS game use as a legitimate training venue, and he 
responded that overall the lOBG students accepted the game as a legitimate training tool 
with some test group subjects going as far as to purchase the Delta Force: Black Hawk 
Down - Team Sabre™ after the experiment in order to replicate the training event at their 
home. During the interview, the TAG said that he observed no negative effects training 
transfer from the game experiment squads. 

The TAG officer felt that the 8-hour exercise time was worth the training benefits, 
but would have preferred it be conducted over a few days rather than all at once. 
Gonducting this over several days was our original plan, but due to scheduling we were 
only allowed one day per squad. The TAG indicated that the exercise was conducted at 
the correct time in the students’ training sequence, between classroom instruction and the 
FTX, and said that given a training plan he would use this type of product for future 
squad training with lOBG students. As a recent veteran of Operation Iraqi Freedom, he 
felt this could further benefit his students if this or other training system integrated 
vehicles and Glose Air Support in order to expose lOBG students to the challenges of 
controlling vehicles and aircraft that could be attached to their unit. 
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When asked what capabilities and equipment he felt could be used to provide 
better training to Soldiers, he indicated the ability to call for indirect fires (mortars and 
artillery), the PEQ-2 weapon-mounted target pointer/illuminator/aiming laser, flares, and 
vehicles. For basic missions such as raids, reconnaissance and conducting ambushes, he 
noted the game could be a good training tool, but cautioned against its use for such 
missions as Stability And Support Operations (SASO) because human interaction is 
essential to training tasks such as these effectively. The TAG liked the ability to easily 
modify the virtual environment using the mission editor. Finally, he said that he would 
like the simulation to support different types of environments to include desert, jungle 
and urban terrain. 

The interviews we held with the participants echoed many of the observations 
made by the TAG. From our test group, these included a feeling of increased awareness 
of their surroundings in the field and comfort level in conducting tactical movement 
through the woods as a member of a squad. One subject noted that “we were forced to 
learn (to maintain 360 degree security while conducting ground tactical movement) 
during the game play, and now it is natural.” 

The test subjects also indicated that they felt more comfortable conducting tactical 
movement through the woods (and maintaining proper spacing and distances) because 
they experienced similar challenges within the game environment. The participants also 
noted the effective employment of binoculars as a positive learning event during the 
conduct of simulated missions. Most of the test group indicated that they would use this 
type of training tool at their unit given the equipment and a training plan, but cautioned 
against its use without unit leadership controlling the exercise. The test group indicated 
that without leadership supervision, this type of training could easily turn into a 
“playground” environment and not a training one. All the test subjects concurred that 
using GOTS games for training has potential, but without clear guidance and supervision, 
its use may not be effective. 

Finally, we asked the test and control groups to discuss effectiveness of 
simulations in general, to train infantry squad collective tasks. Gomments were varied, 

and provided insight to the different learning styles of the subjects from both groups. Of 
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the subjects who found utility in using simulations to train infantrymen, comments 
included: training with simulations “(you) get to see actual movement and contact, and 
get immediate feedback on (your) actions”, and “simulation (use) allowed us to practice 
in more than one scenario, and it saved time. We got done more in less time”. 

Other subjects, from both test and control groups, acknowledged the potential 
benefit of simulation use, but preferred other training methods entirely. One subject 
stated that briefing an operation on a sand table, and then discussing the lesson in a 
classroom was his preferred learning style. He notes that using sand tables “help lowest 
level operators get (a) big picture view”, and that complimenting its use with classroom 
instruction “provides calm, slow-moving environment to properly receive details and 
retain them for live exercise later”. Another subject said that conducting rehearsals in a 
garrison environment (held in an available open field, and conducted without 
ammunition) was better that using simulations. “(Garrison level) physical rehearsals give 
everyone an idea of the actions” and that simulations “will only familiarize the actions”. 
These comments reinforce the importance of trainers at all levels to be aware of the 
individual learning styles of students and plan training events accordingly. 

C. RESULTS OF COTS GAME MISSION EDITOR USABILITY TEST 

To determine the usability of the mission editor tool we used for the experiment, 
we conducted a limited scale usability study upon returning from Fort Benning. The 
scope of the study was to determine how long it would take an inexperienced user to use 
the Delta Force: Black Hawk Down - Team Sabre™ Mission Editor (MED) to recreate 
an environment given only an aerial photograph. The intent was to determine if a novice 
user could be trained in the MED’s use and recreate operationally relevant terrain based 
on current intelligence assets. Due to time constraints, we were not able to set this test up 
in such a way that we could validate the accuracy of the simulated environment. 

The subject for our study was an U.S. Air Eorce Academy cadet on assignment as 
a research intern at the Naval Postgraduate School. The cadet was a pre-Med student and 
had limited computer science skills. Additionally, the cadet indicated that he rarely 
played PC-based video games, and used his issued academy computer for mostly web 
access and word processing. 
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We assigned the cadet a computer workstation loaded with the Delta Force: Black 
Flawk Down - Team Sabre™ software and MED and over the course of the next two 
weeks instructed him to become familiar with the game, complete the MED tutorial that 
came with Delta Force: Black Hawk Down - Team Sabre™, and create two game levels 
of the contemporary operating environment (COE) from web aerial photos of Iraq and 
Afghanistan. We instructed him to record the time it took to become fa mi liar with the 
MED, create the environments, and any lessons learned from the experience. Although 
we acknowledge this study was very limited, it did provide us with insight for our 
recommendations for COTS use. 

We conducted this study in five phases: 

• Phase I: Introduction to software platform 

• Phase: II Initial fa mi liarization with level editor (Mission Editor Manual) 

• Phase III: Selection of aerial photo (Tikrit, Iraq) and Detailed VE design 

using aerial photo 

• Phase IV: VE design of Al-Udeid Airbase 

• Phase V: Einalize and present results 

At the completion of each phase, the cadet briefed us on his findings and we 
provided guidance for the subsequent phase of the study. Eor Phase I, we allowed the 
cadet to play Delta Force: Black Hawk Down - Team Sabre™ on solo missions in order 
to become familiar with the look and feel of the game platform. We felt that this game 
orientation would help him better understand object placement in the 3D game 
environment. Phase II, which took the cadet three hours to complete, involved 
completing the MED tutorial that came with the Delta Force: Black Hawk Down - Team 
Sabre™ game. This tutorial involves creating a simple game level that includes several 
buildings, AI driven OPEOR and vehicular movement orders. Phase III and IV took the 
cadet 27 hours to complete. He created a virtual portion of Tikrit, Iraq, (Eigures 7 and 8), 
and a USAE airfield in Afghanistan. The cadet indicated that actual creation time for 
both levels was about 10 hours, and the remainder was used becoming familiar with the 
MED advanced features. 
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Figure 7. Aerial Photo of Tikrit, Iraq, June 2005 



Figure 8. Virtual Tikrit using the MED 


At the completion of Phase IV of the usability study, our study participant 
identified the following lessons learned and recommendations: 

• Game orientation useful for introducing users to interface requirements 

• Mission Editor manual that came with the game provided adequate 
interface orientation 

• Ean web sites provided a good resource for VE layouts and examples 

• Once trained on the GUI, the interface was easy to navigate and control 

• Eor end users learning to use level editing software, a 3-day training 
course should be developed to achieve a useful proficiency 

• Eor trained users, creation time for robust VEs should range from 2 to 10 
hours depending on the complexity of the environment 
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• Usability, applicability, and simplicity should be considered when 
choosing between different level editing software platforms 


The findings from this small study confirmed many of our impressions of this 
particular level editing software, but we realize more work should be conducted in this 
area. Overall, the findings here provided insight for our recommendations for COTS use 
and selection criteria. 
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VI. RECOMMENDATIONS AND CONCLUSIONS 


A. OVERVIEW 

During our research we found that by properly framing an event infantry leaders 
can use COTS gaming software for specific squad collective training. Due to our limited 
sample size, we were only able to observe subjects conducting a squad attack mission, 
but the elements of what a COTS game can provide to trainers could potentially be 
transferred to additional squad tasks such as conducting reconnaissance, ambushes and 
other combat missions. The key to leveraging COTS software lies with managing the 
expectations of leaders and conducting a thorough task analysis to ensure the trainer 
knows what tasks can and cannot be trained using COTS. These games will not train 
individual tasks such as firing a rifle, but will provide an environment where leaders can 
train cognitive skills such as maintaining security or refining standard operating 
procedures. Training units on these skills is the leader’s responsibility, and it typically 
requires resource intensive live training events. The COTS game alone will not train 
units, but can support unit leaders who leverage the game’s ability to provide robust and 
complex scenarios with minimal resource requirements. 

B. RECOMMENDATIONS FOR TRAINING WITH COTS SOFTWARE 

For leaders planning small unit training events that call for COTS game use, a 
thorough analysis of the task, conditions of the event and standards for evaluation must 
be determined. In the following sections, we make recommendations based on the 
findings from our experiment and our combined 25 years of military experience. These 
recommendations are not meant to be all inclusive, but to provide trainers with a basic 
roadmap for using COTS game software to support infantry training. 

I. Task Analysis 

Once leaders select a unit collective task to be trained, either driven from their 
Mission Essential Task List or war-time mission, a thorough analysis of the task must be 
conducted. In the case of the Army, leaders should use the appropriate ARTEP or field 
manuals to determine performance measures, and associated individual and crew tasks 
that support the collective task. As with live training, supporting subtasks should be 
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trained prior to conducting the collective event. As an example, Soldiers should be 
trained on how to engage a target with their primary weapon. Because COTS games do 
not always model individual skills with precise accuracy, trainers who complete this pre¬ 
training prior to using a COTS game can reduce potential negative skills transfer, because 
the Soldiers will already know the proper way to perform the specific task in the real 
world. 

Tasks that require a very high level of sensory input or fine details that are not 
replicated by the game should be trained in either a specialized training device or during 
a live collective training event. During our task analysis, we separated tasks into two 
groups - alpha and bravo tasks. Our alpha tasks are those which can be replicated in the 
COTS game platform and bravo tasks are those which cannot. For example, an alpha 
task may be engage an enemy with an M16A2 rifle which can be represented well 
enough in the game to support a higher collective task of “React to Contact.” A sample 
bravo task could be search and process an enemy prisoner of war (EPOW). This task 
requires a very high level of tactile feedback and human interaction in order to determine 
the threat the EPOW presents. As stated previously, bravo tasks should be taught at a 
separate event, and leaders should not expect the COTS game to model this task 
effectively. This is not to say that a collective task cannot be trained because a few 
supporting tasks are not modeled well by the game, only that leaders need to plan their 
expected learning outcome accordingly. 

2. Selection Criteria for COTS Software 

Based on the findings from our research, we determined a number of 
recommended benchmarks for selecting a COTS game in order to conduct a squad level 
training event. The benchmarks are divided into three categories: requirements for 
infantry squad collective training, requirements for infantry platoon and Combined Arms 
training, and additional capabilities to support more robust training. 
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To conduct an infantry squad collective training event as we did for our 
experiment, our recommendations for a COTS game platform include, but are not limited 
to the following: 

• PC-based 

• System requirements that support the computer systems common at the 
local unit or installation level 

• First person perspective 

• Multiplayer over a LAN that supports at least 32 players; this would allow 
up to platoon-level training with 2-3 personnel to act as live OPFOR 

• Modeling of modem U.S. weapons and their effects to the extent that it 
supports the overall training event; includes day and night weapon sights 

• An easy to use level (or mission) editor to allow a trained user to quickly 
create a virtual environment that can support the commander’s intent for training 
in any environment (urban, desert, jungle, etc...); has a robust model library 
which includes: people, vehicles, foliage and buildings - each with predefined and 
modifiable attributes 

• The ability to script events and non-player characters with basic AI 
attributes to attack, defend, or other characteristics defined by the commander 

• A heads-up display (HUD) that can help the user overcome the lack of 
peripheral vision (a miniature map that provides local awareness to the user or 
similar tool) 

• A topographical map display of the training area with programmable 
waypoints 

• Player health and ammunition status available through an on-screen HUD 
indicators 

• Spawn or regeneration points to allow players killed during the event to 
rejoin their team at the trainer’s discretion to continue training 

• Configurable player characteristics to allow trainers to assign proper 
weapons and equipment in accordance with duty positions 

• Accurate effects by weapons and environmental conditions on the player 
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• Ability to interact with objects in the environment such as open doors, 
emplaced weapons and drive vehicles 

Although our experiment did not extend beyond a single squad operating in the 
virtual environment without external support, based on our surveys, interviews, and 
experience with COTS game software, we present the following recommendations for 
using COTS to support Combined Arms training:. This list is not intended to be all- 
inclusive, but a guide for trainers selecting COTS for training: 

• Terrain maps of 25 square kilometers; larger for operations involving live. 
Close Air Support 

• Voice over IP or other in-game communication capability to support 
distributed execution of the training event 

• Medical and other Combat Service Support capabilities to include drivable 
vehicles for convoy support or ambulance capability to evacuate wounded players 

• Ability to call for and adjust indirect fire 


Finally, we determined some additional capabilities that would enhance the 
training environment or assist trainers in conducting after action reviews of the event. 
They include: 

• Infrared (IR) aiming/pointing devices 

• The ability to import Military Grid Reference System (MGRS) maps and 
terrain databases into the level editor 

• A tool to allow trainers and leaders to create or import operational 
graphics and display them on the game’s topographical map 

• Flares and illumination pyrotechnics for signaling and night operations 

• Observer view to allow the trainer to see the entire training element or 

move rapidly through the mission area to observe behavior 

• Recording and playback capability for use during the after action review 

• High Level Architecture (HLA) compliance 
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The final recommendation, HLA compliance, would provide the potential for a 
COTS platform to interact with HLA compliant training systems and simulations used by 
the Department of Defense. For example, if the game we used for this experiment was 
HLA compliant, then as a mechanized infantry company trains their Bradley Fighting 
Vehicle crews in the Close Combat Tactical Trainer (CCTT), the infantry squads could 
be conducting concurrent training in the same virtual environment. This could take place 
over a local area network (LAN) that connects the CCTT to the multiplayer game through 
HLA. This would provide a single training event for the entire company team versus 
training the vehicle crews in the CCTT and training the infantry squads separately. 
Finally, HLA capability allows DoD users to be able to leverage future capabilities of the 
commercial market by having a common language for integration. However, it is highly 
unlikely that any COTS game will ever be produced with HLA capability unless the 
military contacts the game developer early in the design process and pays for this 
capability to be added so that the game can be better used for training. 

3. Conduct of the Training Event with COTS Software 

We recommend that trainers considering the use of COTS games as a tool 
approach this simulation event no different than a live event. The only difference in the 
conduct of the event would be the venue. The orders and planning process, preparation 
and conduct of observer/controllers, and AAR are the same as for a live event. 

The training area would be prepared by a trained mission/level editor, creating the 
game level as close as possible to support the commander’s vision. Once the terrain 
model is complete, trainers and leaders should conduct a reconnaissance of the virtual 
training site and the proof the lane to verify the proper sequencing of events involving 
non-player agents and the look and feel of the terrain. 

If the training audience is not fa mi liar with the COTS game controls, we 
recommend that trainers conduct a block of instruction on the game interface and provide 
a virtual training area for the multiplayer squad (or platoon) to conduct rehearsals. 
Trainers should monitor the training events in the game environment, and make spot 
corrections early to prevent horseplay by the trainees using the game. Just as in live field 
training, trainers observe game-play to provide feedback and adjudicate events, casualties 
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and OPFOR actions. At the completion of the game exercise, trainers should conduct an 
AAR using all available tools (including those provided by the game) to spread the 
lessons learned by the users. By keeping the COTS game use approach similar to that of 
a live field exercise, trainers can keep their focus on training infantryman, with only the 
venue being different for our Soldiers or Marines. 

C. CONCLUSIONS AND RECOMMENDATIONS FOR FUTURE 
RESEARCH 

Overall, we found that using COTS software has the potential to be an effective 
low cost and accessible training tool for training infantry squad collective tasks. Despite 
our small sample set, our findings and observations demonstrate some interesting 
possibilities on methods of leveraging COTS software, and implications for use at the 
unit level. While the data recorded from the subject surveys and comments from the 
post-FTX interviews do not conclusively demonstrate that infantry squads can use COTS 
gaming software for training collective tasks, our findings raised many issues that future 
research could address: 

• What is the scope of training that can be accomplished with a COTS game 
product and what is the best method for integrating it into the training plan? 

• Can COTS games provide effective sustainment training to experienced 
units? 

• Can COTS games provide effective training for non-infantry units? 

• Can COTS games provide effective training for the platoon or higher? 

• What effect would training with a distributed (not co-located) squad have 
on training effectiveness? 

• How do individual learning styles effect user acceptance of simulation 
use, COTS or otherwise? 

• What impact would a multiplayer first person simulation, COTS or 
otherwise, on unit level training over an extended period of time? 

• To what extent can intelligent tutoring systems (ITS) remove the 
requirement for an instructor to provide AAR? 
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We hope that future research into this area can begin to explore these areas in 
order to best determine COTS gaming software’s place as a training tool for training 
units. 
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APPENDIX A. OPERATION ORDERS AND GRAPHICS 


A. MISSION ONE: 

I. OPORD #001-05 

CURRENT LOCATION: 2 KM NORTHEAST OE THE TOWN OE KIRKUK AND 
EIRGRESS RIVER AT CAMP SENTINEL 

OPORD #001-05 

SITUATION : ELEMENTS OE AL-EARHEED REBELS HAVE BEEN 

PUSHED OUT OE THE CITY OE MOUSEC DURING THE LAST 48 HOURS DUE 
TO COALITION EORCES SUCCESS. WE EXPECT THESE INSURGENTS TO 
MOVE EURTHER WEST AND ATTEMPT TO SECURE POWER AND RADIO 
STATIONS OE THE TOWN OE KIRTUK (50 MILES WEST OE MOUZEC ALONG 
THE EIGRESS RIVER). THEY ARE ARMED WITH AK-47S, RPGs AND ARE 
PROEIECIENT IN THE USE OE EIELD EXPEDIENT EXPLOSIVES. 

MPCOA: WE CAN EXPECT AL-EARHEED INSURGENTS ON OBJ RADIO 
DEEENDING EROM HASTY LIGHTING POSITONS ORIENTED EAST ALONG 
THE EIRGRESS RIVER. WE EXPECT 6-7 DISMOUNTED INSURGENTS ARMED 
WITH AK-47’S AND RPG EIRES. THEY DO NOT WANT TO BECOME 
DECISIVELY ENGAGED WITH U.S. EORCES, BUT WILL CONTINUE TO EIGHT 
TO BUY TIME EOR THE lEDS TO BE EMPLACED INSIDE THE RADIO TOWER. 

ERIENDLY EORCES: 

• A CO ATTACKS IN ZONE IN ORDER TO SECURE THE TOWN OE 
KIRKUK. 

• PLATOON CLEARS IN ZONE TO DESTROY AL-EARHEED REBELS 
IN AO JAGUAR TO PREVENT ORGANIZED ATTACKS AND LOOTING 
AGAINST THE PEOPLE OE KIRKUK. 

• CIVILIAN: THE MOST OE THE LOCAL POPULATION OE KIRKUK 
ARE IN SUPPORT OP US ACTION, BUT SOME ARE GROWING 
WEARY OP OUR PRESENCE, AND MAY SYMPATHIZE WITH THE 
AL-EARHEED IDEOLOGY. EXPECT REPUGEES TO MOVE EAST 
PROM KIRKUK TO MOUSEC IN ANTICIPATION OP COALOTION AIR 
AND GROUND ATTACKS. THEY WILL USE THE TRAIL NETWORK 
NORTH OP THE EIGRESS RIVER. 

MISSION : 1^^ SQD ATTACKS TO SIEZE OBJECTIVE RADIO IN ORDER 
TO DESTROY ENEMY EORCES AND PREVENT THE DESTRUCTION OP THE 
LOCAL RADIO STATION NET XXXXXXMAR05. 
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EXECUTION: 


CDR INTENT: TO BE SUCCESSEUL, WE MUST: 

1. DESTROY ALL ENEMY EORCES AND EQUIPMENT IN 
ZONE TO PREVENT AN ORGANIZED ATTACK OR 
CAMPAIGN OP TERROR AGAINST THE CITIZENS OP 
KIRKUK 

2. PREVENT CIVILIAN CASUALTIES AND DAMAGE TO 
HOST NATION INPRASTRUCTURE (C2 NODES, 
SCHOOLS, WATER TREATMENT SITES, CHURCHES, 
AND RADIO/TV STATIONS) 

3. RAPIDLY SIEZE OBJECTIVES IN ZONE TO PREVENT 
AN ORGANIZED AL-PARHEED ATTACK AGAINST US 
EORCES 


CONCEPT OP THE OPERATION: WE WILL CONDUCT AN AIR 
MOVEMENT PROM CAMP SENTINEL TO LZ ABLE. PROM LX ABLE, WE WILL 
MOVE 260 DEGREES POR ABOUT 400 METERS TO OBJ RADIO. WE WILL 
CONDUCT A DELIBERATE ATTACK ON OBJ RADIO. ONCE SECURE, WE 
WILL CALL THE PLATOON LEADER WITH A SALUTE REPORT AND REQUEST 
EOD TO CLEAR PUMP OP ANY lEDS (BOOBY TRAPS). 

PIRES: NO PIRES ARE AVAILABLE 

SERVICE SUPPORT : 

REPIT POLLOWING CONSOLIDATION ON THE OBJECTIVE. 
COMMAND AND SIGNAL: NO CHANGE 
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MOUSEC: 50 MILES 


Operational Graphics to OPORD #001-05 



MISSION #1 



B. MISSION TWO: 

I. FRAGO I TO OPORD #001-05 

SITUATION : BECAUSE OE THE SUCCESS OE OUR INITIAE OPERATION, 
WE EXPECT THE AE-EARHEED INSURGENTS TO CONTINUE TO EXEIETRATE 
EURTHER INTO THE COUNTRYSIDE AND DESTROY CRITICAE 
INERASTRUCTURE NODES CRITICAE TO THE TOWN OE KIRKTUK. 

MPCOA: ENEMY INSURGENTS HAVE BEEN DISPEACED SOUTH OE 
THE EIRGESS RIVER AND SIEZED A EOCAE CRUDE OIE PUMPING STATION. 
WE EXPECT THEM TO ATTEMPT TO DESTROY THE OIE PUMPS AND 
RUPTURE THE STORAGE TANKS IN ORDER TO DEEAY U.S. EORCES AND 
TERRORIZE THE EOCAE POPUEATION. WE EXPECT 6-7 DISMOUNTED 
INSURGENTS ARMED WITH AK-47’S AND RPG EIRES. THEY DO NOT WANT 
TO BECOME DECISIVEEY ENGAGED WITH U.S. EORCES, BUT WIEE 
CONTINUE TO EIGHT TO BUY TIME EOR THE lEDS TO BE EMPEACED. 

ERIENDEY EORCES: NO CHANGE 

MISSION : I^^ SQD ATTACKS TO SIEZE OBJECTIVE PUMP IN ORDER TO 
DESTROY ENEMY EORCES AND PREVENT THE DESTRUCTION OE THE 
EOCAE PUMP STATION NET XXXXXXMAR05. 

EXECUTION : 

CDR INTENT: NO CHANGE 

CONCEPT OP THE OPERATION: WE WIEE MOVE 110 DEGREES EOR 
APPROX 240 METERS TO OBJECTIVE PUMP. ON OBJ PUMP, WE WIEE 
ESTABEISH A SBP POSITION NORTH OP PUMP TO SUPPRESS ENEMY 
EORCES, AND USE THE ASSAUET PORCE TO SIEZE OBJECTIVE PUMP. ONCE 
THE ASSAUET PORCE IS ON PUMP, WE WIEE CAPE THE SBP EEEMENT 
PORWARD TO CONSOEIDATE AND REORGANIZE ON THE OBJECTIVE. ONCE 
SECURE, WE WIEE CAPE POD TO CEEAR PUMP OP ANY lEDS. 

SERVICE SUPPORT : 

REPIT AT CHECKPOINT 3 

COMMAND AND SIGNAE: 


NO CHANGE 
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MOUSEC;50 MILES 


2. Operational Graphics to Frago 1 to OPORD #001-05 
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C. MISSION THREE: 

I. FRAGO 2 TO OPORD #001-05 

SITUATION : AL-FARHEED INSURGENTS HAVE BEEN ROUTED, AND 
ARE EOOTING ANY SUPPEIES THEY CAN AND MOVING THEM IN RIVER 
BOATS AEONG THE EIRGRESS RIVER EAST. THEY ARE ATTEMPTING TO 
BEEND IN WITH THE EOCAE POPUEACE MOVING EAST TO MOUZEC. 

MPCOA: ENEMY INSURGENTS ARE ESTABEISHING RIVER EOADING 
POINTS IN OUR AREA TO RAPIDEY MOVE THEIR EOOTED SUPPEIES 
UPRIVER TO THEIR EOCAE STRONGPOINT. WE EXPECT 6-7 DISMOUNTED 
INSURGENTS ARMED WITH AK-47’S AND RPG EIRES. THEY DO NOT WANT 
TO BECOME DECISIVEEY ENGAGED WITH U.S. EORCES, BUT WIEE 
CONTINUE TO EIGHT TO BUY TIME EOR THE lEDS TO BE EMPEACED. 

ERIENDEY EORCES: NO CHANGE 

MISSION : I^^ SQD ATTACKS TO SIEZE OBJECTIVE PIRATE IN ORDER 
TO DESTROY ENEMY EORCES AND SECURE SUPPEIES STOEEN EROM THE 
EOCAE TOWN OE KIRKTUK NET XXXXXXMAR05. 

EXECUTION : 

CDR INTENT: NO CHANGE 

CONCEPT OP THE OPERATION: WE WIEE MOVE TO CHECKPOINT 4 IN 
ORDER TO REPIT ON CE V & VIII. PROM THERE, WE WIEE MOVE 60 
DEGREES EOR APPROX 225 METERS TO OBJECTIVE PRIRATE. ON OBJ 
PIRATE, WE WIEE ESTABEISH A SBP POSITION WEST OP PIRATE TO 
SUPPRESS ENEMY EORCES, AND USE THE ASSAUET PORCE TO SIEZE 
OBJECTIVE PIRATE. ONCE THE ASSAUET PORCE IS ON PIRATE, WE WIEE 
CAPE THE SBP EEEMENT PORWARD TO CONSOEIDATE AND REORGANIZE 
ON THE OBJECTIVE. ONCE SECURE, WE WIEE CAPE POD THE PET EDR IN 
ORDER TO GET SUPPORT PET ON SITE TO EVAC THE EOOTED SUPPEIES 

SERVICE SUPPORT : 

REPIT AT CHECKPOINT 4 

COMMAND AND SIGNAE: 


NO CHANGE 
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2. 


Operational Graphics to Frago 2 to OPORD #001-05 
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D. MISSION FOUR: 

I. FRAGO 3 TO OPORD #001-05 

SITUATION : AL-FARHEED INSURGENTS HAVE BEEN ROUTED, AND 
ARE CONSOEIDATING AT THEIR EOCAE BASES IN ORDER TO REORGANIZE 
AND ATTEMPT TO CONDUCT ATTACKS ON CIVIEIAN REEUGEES MOVING 
WEST TO MOZSEC. 

MPCOA: ENEMY INSURGENTS HAVE SECURED A EOCAE RIVER 
WATCH TOWER TO USE AS A STRONGPOINT AND DISRUPT RIVER TRAEEIC 
MOVING EAST AEONG THE EIRGRESS RIVER. WE EXPECT 6-7 DISMOUNTED 
INSURGENTS ARMED WITH AK-47’S AND RPG EIRES. WE EXPECT AN 
EMPEACED MACHINE GUN IN THE TOWER. THEY DO NOT WANT TO 
BECOME DECISIVEEY ENGAGED WITH U.S. EORCES, BUT WIEE CONTINUE 
TO EIGHT TO BUY TIME EOR THE REBEES TO ESCAPE. 

ERIENDEY EORCES: NO CHANGE 

MISSION : I^^ SQD ATTACKS TO SIEZE OBJECTIVE TOWER IN ORDER 
TO DESTROY ENEMY EORCES AND PREVENT ATTACKS AGAINST 
REEUGEES NET XXXXXXMAR05. 

EXECUTION : 

CDR INTENT: NO CHANGE 

CONCEPT OP THE OPERATION: WE WIEE MOVE TO CHECKPOINT 5 IN 
ORDER TO REPIT ON CE V & VIII. PROM THERE, WE WIEE MOVE 85 
DEGREES EOR APPROX 325 METERS TO OBJECTIVE TOWER. ON OBJ 
TOWER, WE WIEE ESTABEISH A SBP POSITION WEST OP OBJ TOWER TO 
SUPPRESS ENEMY EORCES, AND USE THE ASSAUET PORCE TO SIEZE 
OBJECTIVE TOWER. ONCE THE ASSAUET PORCE IS ON TOWER, WE WIEE 
CAPE THE SBP EEEMENT PORWARD TO CONSOEIDATE AND REORGANIZE 
ON THE OBJECTIVE. ONCE SECURE, WE WIEE CAPE THE PET EDR IN ORDER 
COORDINATE EOR TRANSPORTATION BACK TO CAMP SENTINEE. 

SERVICE SUPPORT : 

REPIT AT CHECKPOINT 4 

COMMAND AND SIGNAE: 


NO CHANGE 
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2. Operational Graphics to Frago 3 to OPORD #001-05 
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E. MISSION FIVE: 

I. FRAGO 4 TO OPORD #001-05 

CURRENT LOCATION: 250 METERS EAST OE THE PERSIAN GULE COAST 
TOWN OE COROZAL 

SITUATION : DUE TO COALITION SUCCESS IN THE NORTH, AL- 
EARHEED INSURGENTS HAVE MOVED SOUTH ALONG THE WESTERN GULE 
COASTLINE AND HAVE INCREASED BOMBINGS, MURDER, MORTAR 
ATTACKS, KIDNAPPING, EXTORTION, HIJACKING, AS WELL AS GUERRILLA 
AND CONVENTIONAL MILITARY ACTION AGAINST THE HOST NATION. WE 
EXPECT THEIR STRENGTH IN OUR DIVISION’S SECTOR TO BE 
APPROXIMATELY 300 TO 1,000 ARMED COMBATANTS AND SEVERAL 
HUNDRED MORE SUPPORTERS, MOSTLY IN RURAL AREAS. IN OUR 
IMMEDIATE AO, WE CAN EXPECT TO SEE 20 - 30 AL-EARHEED INSURGENTS. 

MPCOA: AL-EARHEED INSURGENTS WILL CONTINUE TO DISRUPT 
OEE-SHORE OIL PUMPING OPERATIONS BY CONDUCTING ATTACKS AND 
RAIDING SUPPLIES HEADED EOR THE TOWN OE CORAZOL, WHICH 
SUPPORTS THE OEE SHORE OIL RIG IN OUR AO. WE EXPECT THE REBELS 
TO ESTABLISH ROADBLOCKS ON THE MSR INTO CORAZOL TO RAID EOOD 
AND WATER SHIPMENTS. WE EXPECT 6-7 DISMOUNTED INSURGENTS 
ARMED WITH AK-47’S, .50 CAL MG’S AND RPG EIRES. THEY DO NOT WANT 
TO BECOME DECISIVELY ENGAGED WITH U.S. EORCES, BUT WILL 
CONTINUE TO EIGHT TO BUY TIME EOR LOCAL REINEORCEMENTS IE 
AVAILABLE. 

ERIENDLY EORCES: A CO DEEENDS AO RIG IN ORDER TO SECURE 
THE TOWN OE CORAZOL AND TO PREVENT THE DISRUPTION OP OPE SHORE 
OIL OPERATIONS BY AL-EARHEED REBELS. I^^ PLATOON CLEARS IN ZONE 
TO DESTROY AL-EARHEED REBELS IN AO JAGUAR TO PREVENT 
ORGANIZED ATTACKS AND LOOTING AGAINST THE PEOPLE OP CORAZOL 
AND THE LOCAL OIL PUMPING OPERATIONS. THE MOST OP THE LOCAL 
POPULATION OP CORAZOL ARE IN SUPPORT OP US ACTION IN COLUMBIA, 
BUT SOME ARE GROWING WEARY OP OUR PRESENCE, AND MAY 
SYMPATHIZE WITH THE AL-EARHEED IDEOLOGY. 

MISSION : I^^ SQD ATTACKS OBJECTIVE GARAGE IN ORDER TO 
DESTROY ENEMY EORCES AND SECURE STOLEN SUPPLIES INTENDED EOR 
CORAZOL NLTXXXXXXMAR05. 

EXECUTION : 

CDR INTENT: TO BE SUCCESSPUL, WE MUST: 
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• DESTROY ALL ENEMY EORCES AND EQUIPMENT IN ZONE 
TO PREVENT AN ORGANIZED ATTACK OR CAMPAIGN OP 
TERROR AGAINST THE CITIZENS OP CORAZOL 

• PREVENT CIVILIAN CASUALTIES AND DAMAGE TO HOST 
NATION INPRASTRUCTURE (C2 NODES, SCHOOLS, WATER 
TREATMENT SITES, CHURCHES, AND OPE SHORE PUMP 
PACILITIES) 

• RAPIDLY SIEZE OBJECTIVES IN ZONE TO PREVENT AN 
ORGANIZED AL-PARHEED ATTACK AGAINST US EORCES 

CONCEPT OP THE OPERATION: WE WILL CONDUCT AN AIR 
MOVEMENT TO LZ BEACH. AT LZ BEACH, WE WILL MOVE APPROX 180 
DEGREES POR 400 METERS TO OBJ GARAGE. WE WILL CONDUCT A 
DELIBERATE ATTACK ON THE OBJECTIVE TO DESTROY ENEMY EORCES 
AND SECURE ANY STOLEN SUPPLIES HEADED POR CORAZOL. ALTER WE 
CONSOLIDATE ON THE OBJECTIVE, WE WILL SUBMIT A SALUTE REPORT 
TO RED-1 (PET LDR) AND AWAIT PURTHER INSTRUCTIONS. 

COORDINATING INSTRUCTIONS: 

1. SECURE CHECKPOINT CHARLIE AT LZ BEACH PRIOR TO 
CONDUCTING ATTACK ON OBJ GARAGE. 

2. REHEARSE TACTICAL MOVEMENT AND ACTIONS ON THE 
OBJECTIVE AT THE LIVE EIRE RANGE EAST OP CORAZOL. 

3. SECURE CHECKPOINT DELTA ALTER CONSOLIDATION ON 
OBJ GARAGE. 

SERVICE SUPPORT : 

NO CHANGE 

COMMAND AND SIGNAL: 


NO CHANGE 
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2. 


Operational Graphics to Frago 4 to OPORD #001-05 



104 


FRAGO #4 



F. MISSION SIX: 

I. FRAGO 5 TO OPORD #001-05 

SITUATION : UAV OVERFLIGHT PHOTOS DURING THE LAST MISSION 
HAVE DETERMINED THE LOCATION OF THE LOCAL AL-FARHEED REBEL 
BASECAMP. BATTALION S2 HAS ALSO RECEIVED REPORTS FROM THE 
SIGNAL UNIT ATTACHED TO OUR BATTALION THAT INDICATE THAT THE 
AL-FARHEED REBELS ARE RAPIDLY EXFILTRATING OUT OF OUR AO DUE 
TO THE SUCCESS OF OUR RECENT OPERATIONS. THEY WILL BE LOADING 
SUPPLIES OUT OF THEIR BASECAMP AND ONTO TRUCKS BOUND FOR THE 
COAST. WE EXPECT THEY WILL LOAD PERSONNEL AND EQUIPMENT ON 
CIVILIAN BOATS AND MOVE SOUTH TO REGROUP FOR FUTURE 
OPERATIONS. 

MPCOA: AL-FARHEED REBELS WILL DEFEND THEIR BASECAMP (OBJ 
CLUBHOUSE) IN ORDER TO BUY TIME FOR THEIR FORCES TO FINISH 
LOADING STOLEN SUPPLIES AND EXFILTRATE OUT OF ZONE. WE EXPECT 
6-7 DISMOUNTED INSURGENTS ARMED WITH AK-47’S AND RPC FIRES. 
THEY DO NOT WANT TO BECOME DECISIVELY ENGAGED WITH U.S. 
FORCES, BUT WILL CONTINUE TO FIGHT TO BUY TIME FOR THE LAST OF 
THEIR SUPPLIES TO BE LOADED. 

FRIENDLY FORCES: NO CHANGE 

MISSION : I^^ SQD ATTACKS TO SIEZE OBJECTIVE CLUBHOUSE IN 
ORDER TO DESTROY ENEMY FORCES AND SECURE STOLEN SUPPLIES AND 
EQUIPMENT NET XXXXXXMAR05. 

EXECUTION : 

CDR INTENT: NO CHANGE 

CONCEPT OF THE OPERATION: WE WILL MOVE TO CHECKPOINT 
DELTA IN ORDER TO REFIT ON CL V & VIII. WE WILL MOVE APPROX 100 
DEGREES FOR 320 METERS TO OBJ CLUBHOUSE. WE WILL CONDUCT A 
DELIBERATE ATTACK ON THE OBJECTIVE TO DESTROY ENEMY FORCES 
AND SECURE ANY STOLEN SUPPLIES HEADED FOR CORAZOL. AFTER WE 
CONSOLIDATE ON THE OBJECTIVE, WE WILL SUBMIT A SALUTE REPORT 
TO RED-1 (PET LDR) AND AWAIT FURTHER INSTRUCTIONS. 

COORDINATING INSTRUCTIONS: 

1. SECURE THE CHECK POINT ALPHA AT OBJ CLUBHOUSE AFTER 
CONSOLIDATION. 


105 



SERVICE SUPPORT :REFIT AT CHECKPOINT DEETA PRIOR TO 
MOVEMENT 


COMMAND AND SIGNAE: NO CHANGE 
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2 . 


Operational Graphics to Frago 5 to OPORD #001-05 
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FRAGO #5 




G. MISSION SEVEN: 

I. FRAGO 6 TO OPORD #001-05 

SITUATION : AL-FARHEED REBEES ARE NOW ABANDONING THEIR 
BASES IN OUR AO, AND ARE RAPIDEY ATTEMPTING TO ESCAPE SOUTH 
WITH AS MANY STOEEN SUPPEIES AS THEY CAN. THEY WIEE EXEIETRATE 
SOUTH BY BOAT AND TRUCK (MIXING WITH EOCAES SUPPORTERS IE 
POSSIBEE) TO AVOID US EORCES. 

MPCOA: ENEMY INSURGENTS ARE ESTABEISHING TRUCK EOADING 
POINTS IN OUR AREA TO RAPIDEY MOVE THEIR EOOTED SUPPEIES TO THE 
COAST EOR EXEIETRATION BY BOAT. WE EXPECT 6-7 DISMOUNTED 
INSURGENTS ARMED WITH AK-47’S AND RPG EIRES. THEY DO NOT WANT 
TO BECOME DECISIVEEY ENGAGED WITH U.S. EORCES, BUT WIEE 
CONTINUE TO EIGHT TO BUY TIME THEIR SUPPEY TRUCKS TO ESCAPE. 

ERIENDEY EORCES: NO CHANGE 

MISSION : I^^ SQD ATTACKS TO SIEZE OBJECTIVE EOADING DOCK IN 
ORDER TO DESTROY ENEMY EORCES AND SECURE SUPPEIES STOEEN 
EROM CORAZOE NET XXXXXXMAR05. 

EXECUTION : 

CDR INTENT: NO CHANGE 

CONCEPT OP THE OPERATION: WE WIEE MOVE TO CHECKPOINT 
AEPHA IN ORDER TO REPIT ON CE V & VIII. WE WIEE MOVE APPROX 210 
DEGREES EOR 320 METERS TO OBJ EOADING DOCK. WE WIEE CONDUCT A 
DEEIBERATE ATTACK ON THE OBJECTIVE TO DESTROY ENEMY EORCES 
AND SECURE ANY STOEEN SUPPEIES HEADED EOR CORAZOE. APTER WE 
CONSOEIDATE ON THE OBJECTIVE, WE WIEE SUBMIT A SAEUTE REPORT 
TO RED-1 (PET EDR) AND AWAIT PURTHER INSTRUCTIONS. 

COORDINATING INSTRUCTIONS: 

1. SECURE THE CHECK POINT BRAVO AT OBJ EOADING DOCK 
APTER CONSOEIDATION. 


SERVICE SUPPORT : 

REPIT AT CHECKPOINT AEPHA PRIOR TO MOVEMENT 
COMMAND AND SIGNAE: NO CHANGE 
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Operational Graphics to Frago 6 to OPORD #001-05 
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FRAGO #6 




H. MISSION EIGHT 


I. FRAGO 7 TO OPORD #001-05 

SITUATION : AL-FARHEED REBEES IN OUR AO ARE HAVE BEEN 
EUEEY ROUTED AND ARE RAPIDEY ESCAPING SOUTH IN SECTOR WITH AS 
MANY STOEEN SUPPEIES AND PERSONNEE THEY CAN. THEY ARE 
EXEIETRATING BY BOAT AND TRUCK SOUTH AEONG THE COAST IN 
ORDER, AND THEN WIEE PUSH INEAND TO THEIR RURAE BASE CAMPS. 

MPCOA: AE-EARHEED REBEES ARE ESTABEISHING COASTAE 

EOADING POINTS IN OUR AREA TO RAPIDEY MOVE THEIR EOOTED 
SUPPEIES SOUTH AEONG THE COAST IN ORDER TO REEIT EOR EUTURE 
OPERATIONS. WE EXPECT 7-9 DISMOUNTED INSURGENTS ARMED WITH 
AK-47’S, .50 CAE MG AND RPG EIRES. THEY DO NOT WANT TO BECOME 
DECISIVEEY ENGAGED WITH U.S. EORCES, BUT WIEE CONTINUE TO EIGHT 
AEONG THE SHORE TO BUY TIME EOR THEIR MAIN SUPPEY BOATS TO 
ESCAPE. 

ERIENDEY EORCES: NO CHANGE 

MISSION : I^"^ SQD ATTACKS TO SIEZE OBJECTIVE MARINA IN ORDER 
TO DESTROY ENEMY EORCES AND SECURE SUPPEIES STOEEN EROM THE 
EOCAE TOWN OE CORAZOE NET XXXXXXMAR05. 

EXECUTION : 

CDR INTENT: NO CHANGE 

CONCEPT OP THE OPERATION: WE WIEE MOVE TO CHECKPOINT 
BRAVO IN ORDER TO REPIT ON CE V & VIII. PROM THERE, WE WIEE MOVE 
200 DEGREES EOR APPROX 225 METERS TO OBJECTIVE MARINA. WE WIEE 
CONDUCT A DEEIBERATE ATTACK ON THE OBJECTIVE TO DESTROY 
ENEMY EORCES AND SECURE ANY STOEEN SUPPEIES HEADED EOR 
CORAZOE. APTER WE CONSOEIDATE ON THE OBJECTIVE, WE WIEE SUBMIT 
A SAEUTE REPORT TO RED-1 (PET EDR) AND AWAIT PURTHER 
INSTRUCTIONS. 

SERVICE SUPPORT : 

REPIT AT CHECKPOINT BRAVO PRIOR TO MOVEMENT 

COMMAND AND SIGNAE: 


NO CHANGE 
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APPENDIX B. SUBJECT SURVEYS 


DEMOGRAPHICS SURVEY: 


Participant Dwiographics. CipariaiKa. aixl Training Omstionnair* 
Coieciwt Tratiing EKpanmanI 

Sponsored by tfte MOVES ineMute Hovel Poetgraduete School 


Subject« 
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0 

o 
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(le 0-i» 
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10 Current Service Component 

o 

o 

0 
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o 

o 

0 0 

Militarv Exaerlence 



Ho 
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o 

o 
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13 In what tKsnch did you serve'’ O 

o 

o 
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NG 
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o 

o 

o 
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o 

o 
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lie 310) 
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.VM, 
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o 
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0 
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O 

O 
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o 
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Compam 

19a rr*Yes* at what level wee your most recent OPORD? 

O 

o 

o 
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Simulations Exoerience 


Daily 

Weekly 

MontNy 

Never 


20 In the pear MO yearY howoAtn M you play video oerne«> 

0 

0 

0 

0 



Daily 

Weekly 

Monitey 

Never 


21 How often do you now play video oames> 

0 

0 

0 

0 
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>*or>e 
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0«iei 
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0 
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0 

o 
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Oa«y 
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o 

0 

o 

0 

o 



Never 

Rarely 
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Individual Trainina Assessment 
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0 

0 

0 

0 



under enemy fire as a member of a squad? 





34 Hoa-confktsntare you in vour trainna levai to oqninivircilt 1 






tacticalrrvavemeni orders widerenemv Are? fie 'AloTia 















unsjn 



VarvContiaire 



34a To my baitle buddy Q 

0 

o 

0 

o 



34b To my tire warn Q 

0 

0 

0 

0 



34c To my squad 0 

0 

o 

0 

o 



Vaiv Unauie 

Umtre 

NeWrai 

Conhctanl 

VervCorrfOM* 


33 Hewtanftilenlaieveu i vow tfirnraieveHo uae*«itUe»eepvgftB 0 

0 

0 

0 

0 



manauvar agamu an anamy poaaon> f 



■ 



Vfiy i^naury 

LHiiura 

Nautrai 

Connaam 

v'arvComiatm 


36 HOW conAOam ai* you (n youf trsuing lovai lo um avaaabto 0 

0 

0 

0 

0 



canceaWwnl tamaneuveranaral an enamv Motion'* t - 






veiv Unsure 

umwe 

NeWrW 

Confidem 

VervConnaert 


37 HowconrKtentareyou in your trarMnetev«ltDscan.<iei6ct 0 

0 

0 

0 

o 



and idonbfy enemy pos*ons? ^ 







urmaa 

Naotrai 

ConMant 

vary Connaam 


3$ How oonMont are you in your tramnQ level to submit tactical 0 

0 

o 

0 

0 


"t repofto (ACE SALT, SALUTE)? . ---' 



- 


Sauad Trainina Assessment 


Vaiy uoaure 

Urawa 

Nautrai 

Conftaant 

varvConnasm 


39 How oonMertI we you in your aouad'a traeimg level to O 

0 

o 

0 

0 


) 

maneuver under enemy fire? ; 

.K 






' Vaiv Unsure 

UDan 


ConfMsni 

VarvCanMer* 


40 How continent are you in your aouatf'# traemg level to 0 

0 

o 

o 

0 



coordmate drrect fires ao«nst an enemy pos*on? ^ 





41 How confktont are vou in your souaU’* trwnna level to move 1 — 

— . 

— 

- 

.. _ 


lacUMl^ viivunvn 

Joaura 

NauCrat 

Conttaant 

vary Connaam 



4ie TraveUng Q 

0 

0 

0 

0 



41b Traveling Overwatch Q 

0 

0 

0 

0 



41c Boundng Overwetch Q 

0 

0 

0 

0 



Very unsure 

unswa 

Nautrai 

Connaam 

varvConnaerc 


42 Hew conAdeffl w« you vi yevr leuaU^ tranng level te um avataCIa O 

0 

o 

0 

o 



tonef to trwwwer agaiwt an enemy agudon'* j 






Vary Unaure 

linatra 

Nmrfrai 

Connaam 

varvConMM« 


43 HowcontVtemvreyounrewraouBUetraeingieveiloueeevwWte 0 

0 

0 

0 

0 



concwfimfnt to nwiSMar aganal an anamy poMion? j 






vary Unaura 

Untwa 

Nautrai 

Connaam 

varv Connaam 


44 HowconfktenI areyou in youraquad'a tramng level to scan 0 

0 

0 

0 

0 



detect and identilv an enemy powfton? -- 






■ • • • •• Very Uneure 

uniwt 

Neutral 

ConWem 

VervCoritBert 


45 How coriMent are you in your aquad'a trarano to 0 

0 

0 

0 

0 



conduct a Squad Attack? i 
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B 


POST MISSION SURVEY (TEST) 





Subtect a 







Mission 

- START HERE 




0*1 a o' 1 • i:: i 1 = Qxr-ftrtei^ poor usar, '■ s avarsgt usar. iu s ftipcn usftr rjito .>3urM!<r on your 'nas:ftfy 

0 the simulstion't conUots dumg this mission (how easy was t for you to do what you wanted in the simulation); 




ATeemAR 

ATmGR 

ATmRM 


1 What was your poMon in this mission'^- 0 

0 

o 

0 

o 



6 loam Ldi 

0 

BJflKLAB 

o 

aTffiGR 

0 

BTmBM 

o 


^uad Ifsdsf 


ATaamAR 

ATm GR 

ATmRM 

NoOflt 

Who did you rnaintain visual contact w«h ^ 

0 

0 

0 

0 

0 


6TMfn Lflf 

6T«ao)AR 

BTmOR 

BTmRM 



0 

o 

0 

o 




Yes 

»0 

LOanTKiMMf 


) Were you wounded dursig this rmssion? 


0 

0 

0 




Yes 

No 



4 Did you die during this mssnn'? 


0 

0 





111 




5 Did you sae tha anarrry dunng ttits m«sion? 


0 

0 





Yes 

No 


6 Did you engage tha enemy?. 


0 

0 

l‘•NO" proceed tO»7 




Na 



6a Ware you able to assess your effects on the enemy? 


0 

0 




Eggr 

£21 

Coen 

v*«vrvan 

E.>£eter< 

How wed did you feel you mamtaned your posapn in the 

7 formabon? 

o 

o 

o 

o 

o 



Sousd LOR 

TM LOR 

Buddy 


e Whose commands cM you most frequensty hear? 


0 

0 

0 



£02 

£21 

Goon 

very Good 

D^cetert 

9 ReM your aNity to communicale wdh your superior. 

0 

o 

0 

o 

0 


£92 

£21 

Good 

Very Good 

Ekcafert 

10 Rale your aNKy to communicale wSh your teammates 

o 

o 

o 

o 

o 


Poor 

£21 

Good 

vt'vCtaM 

Ertetert 

11 Rale your aWity to communicale wet) your subordmaies 

0 

o 

0 

o 

o 

For queslwns 12' IB. If you run out at roam. piMM conlini,M on the back ol the page. 

What did you use in the mission that you 

avatiaOie? 

u What eouKintent did you not have available m 

tha simulation that you normally would have? 


14 In terms of the simulabon controls, 1 was 

challenged by 


1ft In terms of the wtual environment t was 

challenged by: 


Mission 

- STOP HERE 
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POST SIMULATION SURVEY (TEST) 


Post-Simulation Questionnaire 



A Teem lit 

ATeamAR 

ATmOR 

ATm RM 

1 Whal frouad poMionft did you ho(d dumg me exercise? 

o 

o 

o 

0 

o 

(f IN VI sN ttiat ^ly) 


& Teem Ldr 

B Team AR 

BTmGft 

8 7m RM 



o 

o 

0 

o 


Poor 

£m 

Good 

Vert Good 

frTTlin 

2 How weN did you understand four duties & responsiMiiies? 

o 

o 

o 

0 

o 


Poor 


Good 

Wen 

VervWsI 


o 

o 

o 

0 

0 








esfic 

Eai 

Good 

WeH 

VfnrVW 

4 How weN did you understand the o4h«r squott /nembofs dubes 

o 

o 

o 

o 

o 

and lesponsibdibes? 


east 

Eiu 

Good 

ttfill 

Very WeS 


o 

o 

o 

o 

o 

■ 


Venr Unsure 

Unatee 

Nemra 

Oonaoem 

YmCflnfeim’ 

6 How conhdent are vow vi vour trainvia tevcl In maneuver under 

o 

o 

o 

0 

o 

er>erny tire as a membei of a squad? 

7 How conhdent are you m your trairang level to oommuiicale 

tactical movement orders under enemy fire? 

Vent Unsure 

UOHCS 

Neiisa 

CooSdenl 

Very Confident 

7a To my battle buddy 

o 

o 

o 

o 

o 

7b To my fire team 

o 

o 

o 

0 

o 

7c Tomys(|uad 

o 

o 

o 

0 

o 

d How conhdeni aie vou m your trairwiQ level lo use avaiiablecover 

Very Unsure 

Uimrt 

Neuea 

ConSdent 

Verv CorrtKtent 

: Id maneuver against an enemy posibon? 

0 

o 

0 

0 

o 

d How conhdem are you ai your traning levet lo use avaiaue 

Vent Unsure 

Unft>#e 

Nemra 

ConSram 

Verv CcnMent 

ooncealfnent lo maneuver aaaxtst an enemv oosibon?: 

o 

o 

o 

o 

o 

10 HowccvifUenlareyouviyour’ Iramirtg level lo scan, deleci and 

Vert Unsure 

Unetre 

Nouea 

ConAdent 

Verv ContKtent 

idenlify an enemy poMion? 

o 

o 

o 

0 

o 

11 How conhdenl are you viyour framing level lo subml tactical 

Vent Unsure 

Unwae 


ConSawii 

Very CcftSdent'. 

reports (AC€. SALT. SALUTE)?: 

o 

o 

o 

0 

o 

12 How conhdenl are you vi yowr aquad^ Iramirtg level to maiemver 

Very Unsure 

Unt>ve 

Neuira 

ConAdent 

Very Corttbeni 

ureter enerrry fire? 

0 

0 

0 

0 

o 

13 How conMeni are you m yow equarfa traming level lo coordirute 

Vent Unsure 

Uneiae 

Neosa 

ConAdertl 

YmCgiWMl 

direct hres against an enemy poaNion?. 

o 

o 

o 

o 

o 

14 Howconhdentareyouviyouraquad'a iranino level lo move 

tactically m the loNavnng movement formabons? 

Venr Unsure 

Une>#e 

Neulrel 

ConAdsm 

Verv Ccnfctent 

14a Traveling 

o 

o 

O 

0 

o 

14b Traveling Overwalch 

o 

o 

O 

0 

o 

14c Bounding Overwatch 

o 

o 

o 

o 

o 

15 Hmv ccoAdeni are you m yovr aquatf's tracing level to use 

Vert Unsure 

UosiCt 

Nemta 

ConAdent 

Very Ccnfctent- 

available poyer lo maneuver against an enemy powbon? 

o 

o 

o 

o 

o 

10 How ccnhdenl are you fi yowr aquatf'a traning level to uce 

Vert Unsure 

Unsure 

heiiira 

ConAoent 

Very CorvSdent 

available corKiaalmenl lo maneuver aoantl an crrerrrv ooiilnn? 

0 

o 

o 

0 

0 

17 How oonMem are you in your squad's framing level to scan. 

Vent Unsure 

Uns»#p 

tUidol 

ConAdem 

Verv ConfcdenI 

delect and idenWy an enemy position? 

o 

o 

o 

0 

o 

18 How conhdent are you m your oqu»iS% traning level to conduct 

very Unsure 

Uracre 

Nemra 

ConAdent 

very Contbent 

actioos on contact n a FTXiLFX? 

o 

o 

o 

o 

o 


Post simulation survey (page 1 of 2) 
117 




















































Rale 1h* simiiabon's atMlily to provide audto cues that alowed 
' vou Id identifV the tocalion otenemv Rret^ 

Poor 

0 

Par 

o 

Good 

o 

wei 

o 

VerrWeS 

0 

20 Rale the iimuiation'^ ability to provide visua/ cues that slowed 
you lo Identity the tocabon o( enemy ftres? 

Poor 

o 

En 

O 

Good 

o 

Wetl 

o 

Very Wee 

O 

21 Rale the simulabon's sMity to provide audio cues that aiowed 
' vou Id determine the locabon ofvoui souad members? 

Poor 

o 

Far 

o 

Pood 

o 

Well 

o 

Vsrvwel 

o 

22 Rate the simutsbon'e ability to provide v/euar cues diet aiowed 
you lo determine the locabon otyptif flfliaff fnttm&ftff? 

Bass 

O 

Eat 

O 

Qood 

o 

Writ 

o 

Very Wei 

O 

23 Rate the simulation exercise's effectiveness lo tramyou to 
. oonduet a Squad Attack? 

Poor 

0 

Effi 

o 

Good 

o 

Wetl 

0 

vent Wee 

0 

24 Rale the simuiabon exerctse's effectiveness lo Iranyour squad 
^ lo conduct a Squad Attack? 

Poor 

o 

Ew 

o 

Good 

o 

Wetl 

0 

Very Wed 

0 

2S Rate the simutabon exercne's effectiveness to providejrou wffh 
sduabonai awareness^ <How 1 see mvsetf A the enemvl 

Poor 

o 

Em 

O 

Good 

o 

wetl 

0 

very Wes 

0 

^ Rate the statement's descnption of yoia experience Curing ffw 
exerctso. 1 Ml tike my actions m the wtuaf enyiionmenf had no 
coneequences * 

iiorTfue 

o 

gprnyohji This 

o 

Neutral 

o 

loei 

0 

Very True 

o 

27 Rale the statement's description of yoia experience Curing tfMis 
exercfse. iMibkel imm pteymg a game ' 

Wot True 

o 

Some*ttelTrue 

o 

Neutrat 

o 

IQ» 

0 

Very True 

o 

28 Rate the otatomerTfs description of your experience: CunngttMs 
exercise, r tefr Ure f was conducting framing ' 

Not True 

o 

3orne«>hal True 

o 

Nemre 

o 

Isa 

0 

Very True 

0 

For questions 29 • 31 please continue on the back of the page rf you run out of room. 




jg This Iraning would be more effective if H included 








3Q If 1 could add anything to ths traireng. 1 woM add 



If 1 could add anything to this sanulabon i would add 






-f 


32 Any addibonai comments 
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D 


POST FIX SURVEY (CONTROL) 


Post Squsd FTX Survsy 
Collective Training Experiment 

Sponsored by the MOVES Institute. Naval PostQraduate Sct>ooi 


Subtect # _ 

(Votf uifKl vifTtwr • t>» tm ttna Mart al yOM M Man* 
•m IMI 4 40* tf rwr UN 

Porewancfe a.TwiR«T<w «2>«»CTWi»'R«M|79r< 


Muttipie Choi>ce ouestrons Piface a check tn arcse. for exempie 


TFTT 


o 



Fift-in quesaons. use space provrdeO (£nter "N/A • ff not appiicatle) Ranger School 


FTX Individual Assessment Survey 


What squad positions did you hold during the blank fire exercise? 

Souad Leader 

0 

Team Ldr 

0 

AR 

0 

QR 

0 

RM 

0 


2 How well did you understand your duties & responsibilities?: 

Poor 

Call 

Good 

Well 

Very Well 


0 

0 

0 

o 

0 

1 Dunrig (he FTX missions, on average, how well did you feel you 

Poor 

Eiit 

Good 

Welt 

VsrvWeH 

understood the mission’.' 

0 

o 

o 

0 

0 


hO’A well (M your traming prepare you to... 

^ maneuver under enemy fire as a member of a squad?, 
fj communicate tacbcai movement orders to your... 


5d 

{AM positions) 

battle buddy’ 

Gb 

(Team Leaders) 

fire team? 

5c 

(Squad Leader) 

squad? 

G 

use availabte cover to manuever agamst the eriemy? 


7 

use avaiiabie concealment to manuever against the enemy? 


8 

scan, detect and identify the enemy? 



submit tacbcai reports (ACE. SALULTE)? 



o 

o 

o 

o 

o 

o 

o 

o 

o 


Eii: 

o 

o 

o 

o 

o 

o 

o 

o 

o 


I'uw well did you... popf 

10 maneuver under enemy fire as a member of a squad? O 

11 communicate tacbcai movement orders to your.. 0 

11a (Al positiorts} battle buddy‘> O 

11b (Team Leaders) fire team? 0 

11c (Squad Leader) squad? O 

12 use available cover to manuever agamst the enemy? 0 

13 use available ooncealmenl to manuever against the enemy? O 

14 scan, detect and Identify the enemy? 0 

15 submit tacbcai (ACE. SALULTE''^ 0 


Fair 

o 

o 

o 

o 

o 

o 

o 

o 

o 


FTX Squad Assessment Survey 


l-ktw well did you feel your squad uixlerstood the mission ?.' 


1 7 How well did you understar>d the other squad members' dubes and 
responsibilities’ 


EBQ[ 

o 


Poor 

o 


Fair 

o 


Fair 

o 


Good 

o 

o 

o 

o 

o 

o 

o 

o 

o 


Good 

o 

o 

o 

o 

o 

o 

o 

o 

o 


S«st 

o 


Good 

o 


W«ll VetvWell 

o o 

o o 

o o 

o o 

o o 

o o 

o o 

o o 

o o 


VSWI Vervwoll 

o o 

o o 

o o 

o o 

o o 

o o 

o o 

o o 

o o 


W«« Vetvw oll 

o o 


Won V«rv W«ll 

o o 
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How well did your trailing prepare your squad to. Psqt Eii; Good w<i v«fv w»>i 

18 maneuver under enemy Tire? 0 0 0 0 0 

19 coordinate direct fires against the enemy? 0 0 0 0 0 

20 n>ove tactically in the following formations. 0 0 0 0 0 

20a trave«ing? 0 0 0 0 0 

20Ci travelling overwatch? 0 0 0 0 0 

20c boundir>g overwatch? 0 0 0 0 0 

21 use available cover to mar>uever against the enemy? 0 0 0 0 0 

22 use available concealment to manuever against the enemy? 0 0 0 0 0 

23 scan, detect and identify the enemy? 0 0 0 0 0 

24 conduct actions on contact? 0 0 0 0 0 


How well did your squad gjn Good W«H Vrv W«ll 

25 maneuver under enemy fire? 0 0 0 0 0 

26 coordinate direct fires against the enemy? 0 0 0 0 0 

27 move tactically in the following formations. 0 0 0 0 0 

27a traveSing? 0 0 0 0 0 

27b travelling overwatch? 0 0 0 0 0 

27c boondmg ovenvatch'J 0 0 0 0 0 

28 use available cover to manuever agamst the enemy? 0 0 0 0 0 

29 use available concealment to manuev^ against the enemy? 0 0 0 0 0 

30 scan, detect and Identify the enemy? 0 0 0 0 0 

31 conduct actions on contact'^ 0 0 0 0 0 



32 How well did the trainmg you received in the previous weeks prepare you 
to c^mand and control a squad? 

Poor Fair Good Walt Varv Wall 

0 O 0 0 0 


33 

Did you feel you needed more training prior to the Live Fire Exercise? 

Yes Ng 

0 0 1 IIW. proc*«dlo#37 1 


34 

If 'yes*', how much more trainino fin hours)? 





35 If field framing a not posable, rank the following choices for alternate 
trainmg (1 • first arxj 5 * last) 


Ciaserooin Sand Table R ehearsal Smulatton 


Other uramino tvoe and «g rank) 


Explain your #1 arxJ #2 ranking for #43 


37 How well did the trainmg you received m the previous weeks prepare you 
to assess the situation when <n contact? 


Poor 

o 


Fair 

o 


Good 

o 


Won 

o 


Vorv Won 

o 
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POST FTX SURVEY (TEST) 


Post SqiMid FTX Survty 
Collective Training Experiment 

Sponsored by the MOVES Irtstitute. Naval Postgratluate School 


Subject P _ 

(Voi« ■utter « te tel MteTB «l you tel ten* 

•rd Itfl 4 tf y«ur SSN 
a.TMlRte«0. 


MuWple Chotce questions Piece e cfteck m arcle. for exempte 



Filt-m questions, use space provided (Enter y>t/A * if not appticabte) 


Ranger School 


FTX Individual Assessment Survey 


1 

What squad positions did you hold dunr>g the blank fve exerose? 

Souad Leader 

0 

Team Ldr 

0 

OIS 

m 

0 

oil 


2 How well did you understarxJ your duties & responsibilities?; 

Poor 

Fair 

Good 

Wall 

Very Well 


0 

0 

0 

0 

o 

3 Dunr>g (he FTX missions, on average, how well dtd you feel you 

ESS! 

Eait 

Good 

Well 

Vary Well 

understood the mission?; 

0 

0 

0 

o 

0 


well cM your training prepare you to. . 

4 maneuver under enemy fire as a member of a squad?. 
^ communicate tactical movement orders to your.. 


5j 

(Al pOSltiOTiS) 

bathe buddy? 

Gb 

(Team Leaders) 

fire team? 

5c 

(Squad Leader) 

squad? 

G 

use available cover to manuever agaaist the enemy? 


7 

use available concealment to manuever against the enemy? 


B 

scan, detect and identify the enemy? 


H 

submit tactical reports (ACE. SALULTE)? 



Poor 

o 

o 

o 

o 

o 

o 

o 

o 

o 


F»ir 

o 

o 

o 

o 

o 

o 

o 

o 

o 


0 0 0 

0 0 0 

0 0 0 

o o o 

0 0 0 

0 0 0 

0 0 0 

0 0 0 

o o o 


»ow well (M you... 


10 

maneuver under enemy fire as a member of a squad? 


11 

communicate tacbcal movement orders to your.. 


11a 

(Al positions) 

bathe buddy? 

11b 

(Team Leaders) 

fire team? 

lie 

(Squad Leader) 

squad? 

12 

use available cover to manuever agamst the enemy? 


13 

use available concealment to manuever against the enemy? 


14 

scan, detect and identify the enenty? 


15 

submit tactical 'upoits ';ACE. SALULTE)’’ 



Poor 

o 

o 

o 

o 

o 

o 

o 

o 

o 


Ear 

o 

o 

o 

o 

o 

o 

o 

o 

o 


FTX Squad Assessment Survey 


Cssil 

O 

o 

o 

o 

o 

0 

o 

o 

o 


VVSU Vetv Well 

o o 

o o 

o o 

o o 

o o 

o o 

o o 

o o 

o o 


How well did you feel your squad urnferstood the mission?; 


Poor Eiir QsM Well Vary Well 

0 0 0 0 0 


How well did you understand the other squad members' duties and COSU Eau fioad very Well 

responsibilities? 0 0 0 0 0 
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How well did your trailing prepare your squad to. Psqt Eii; Good w<i v«fv w»>i 

18 maneuver under enemy Tire? 0 0 0 0 0 

19 coordinate direct fires against the enemy? 0 0 0 0 0 

20 n>ove tactically in the following formations. 0 0 0 0 0 

20a trave«ing? 0 0 0 0 0 

20Ci travelling overwatch? 0 0 0 0 0 

20c boundir>g overwatch? 0 0 0 0 0 

21 use available cover to mar>uever against the enemy? 0 0 0 0 0 

22 use available concealment to manuever against the enemy? 0 0 0 0 0 

23 scan, detect and identify the enemy? 0 0 0 0 0 

24 conduct actions on contact? 0 0 0 0 0 


How well did your squad gjn Good W«H Vrv W«ll 

25 maneuver under enemy fire? 0 0 0 0 0 

26 coordinate direct fires against the enemy? 0 0 0 0 0 

27 move tactically in the following formations. 0 0 0 0 0 

27a traveSing? 0 0 0 0 0 

27b travelling overwatch? 0 0 0 0 0 

27c boondmg ovenvatch'J 0 0 0 0 0 

28 use available cover to manuever agamst the enemy? 0 0 0 0 0 

29 use available concealment to manuev^ against the enemy? 0 0 0 0 0 

30 scan, detect and Identify the enemy? 0 0 0 0 0 

31 conduct actions on contact'^ 0 0 0 0 0 



32 How well did the trainmg you received in the previous weeks prepare you 
to c^mand and control a squad? 

Poor Fair Good Walt Varv Wall 

0 O 0 0 0 


33 

Did you feel you needed more training prior to the Live Fire Exercise? 

Yes Ng 

0 0 1 IIW. proc*«dlo#37 1 


34 

If 'yes*', how much more trainino fin hours)? 





35 If field framing a not posable, rank the following choices for alternate 
trainmg (1 • first arxj 5 * last) 


Ciaserooin Sand Table R ehearsal Smulatton 


Other uramino tvoe and «g rank) 


Explain your #1 arxJ #2 ranking for #43 


37 How well did the trainmg you received m the previous weeks prepare you 
to assess the situation when <n contact? 


Poor 

o 


Fair 

o 


Good 

o 


Won 

o 


Vorv Won 

o 
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1_1 

What effect dkj the simulation traming have on .. 

|38 your performance? 

0 

0 

fiSLSJm 

0 

Poaerve 

0 

vary Well 

0 

39 

your ability to assess the situation when m contact? 

o 

o 

o 

o 

o 

L 

your ability to maneuver under enemy Tire as a member o< a squad? 

0 

0 

0 

0 

0 

kour ability to communicating arxl recerve tactical movement orders under 
ffire 






41 

to your battle buddy? 

0 

0 

0 

0 

0 

42 

to your fire team? 

O 

0 

0 

0 

0 

43 

to your squad? 

O 

0 

0 

0 

0 

44 

your ability to use available cover to maneuver against the enemy? 

0 

0 

0 

0 

0 

45 

your ability to use available conoealnwnl to maneuver agairtst the 
enemy? 

O 

0 

0 

0 

0 

46 

your ability to scan, delect and wentify the enemy? 

0 

0 

0 

0 

0 

47 

your undefstandir>g of squad manuevers? 

0 

0 

0 

0 

0 

46 

your squad’s pert0fmance7 

0 

0 

0 

0 

0 

49 

your squad’s ability to communicate information? 

0 

0 

0 

0 

0 

50 

your squad’s abriity to control direct fires’^ 

o 

0 

0 

0 

0 

51 

your squad's abiirty to marwuever under fire? 

o 

0 

0 

0 

0 

1_ 


Post FTX survey (test) (page 3 of 3) 
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APPENDIX C. FM 7-8 REACT TO CONTACT BATTLE DRILL 


TASK: React to Contact (Platoon/Squad) (07-3-D9103) 

CONDITIONS: The platoon/squad is halted or moving. The enemy initiates fires on the 
platoon/squad with an individual or crew-served weapon. 

STANDARDS: The unit returns fire immediately. The unit locates and engages the 
enemy with well-aimed fire and causes at least one enemy casualty. The leader can point 
out at least one-half of the enemy positions and identify the types of weapons (such as 
small-arms, light machine gun). 

SUPPORTING INDIVIDUAL TASKS: See also Appendix A, "Individual Task-to- 
Drill Matrix." 


References 


Task Number 


Task Title 


STP21-1-SMCT 071-311-2007 

071-311-2027 

071-311-2029 

071-311-2127 

071-311-2129 

071-311-2130 

071-312-3029 

071-312-3031 

071-325-4407 

071-326-0502 

071-326-0503 

071-326-0511 

071-326-0513 

181-906-1505 

181-906-1505-A 

STP 7-11BC1 -SM-TG 071 -052-0006 


071-054-0004 

071-312-4027 

071-326-0501 

STP7-11BC24-SM-TG 071-052-0006 


071-054-0004 

071-312-4027 

071-326-0501 

071-326-5611 

071-326-5630 

071-420-0005 

STP 7-11BCHM1 -SM 071 -052-0006 


EngageTargets with an M16A1 or M16A2 Rifle 

Load an M16A1 or M16A2 Rifle 

Correct Malfunctions of an M16A1 or M16A2 Rifle 

Load an M203 Grenade Launcher 

Correct Malfunctions of an M203 Grenade 

Launcher 

Engage Targets with an M203 Grenade Launcher 
Correct Malfunctions of an M60 Machine Gun 
Engage Targets with an M60 Machine Gun 
Employ Hand Grenades 
Move Under Direct Lire 

Move Over, Through, or Around Obstacles (Except 
Minefields) 

React to Llares 

Select Temporary Lighting Positions 

Conduct Combat Operations According to the Law 

of War 

Conduct Combat Operations According to the Law 
of War 

Engage Targets with an M47 Medium Antitank 
Weapon 

Engage Targets with an Ml36 Launcher 

Load an M249 Machine Gun 

Move as a Member of a Lire Team 

Engage Targets with an M47 Medium Antitank 

Weapon 

Engage Targets with an Ml36 Launcher 
Load an M249 Machine Gun 
Move as a Member of a Lire Team 
Conduct the Maneuver of a Squad 
Conduct Movement Techniques by a Platoon 
Conduct the Maneuver of a Platoon 
Engage Targets with an M47 Medium Antitank 
Weapon 
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References 


STP 7-11BCHM24-SM-TG 


Task Number 
071-054-0004 
071-312-4027 
071-326-0501 
071-326-5611 
071-326-5630 
071-420-0005 


Task Title 

Engage Targets with an Ml36 Launcher 
Load an M249 Machine Gun 
Move as a Member of a Lire Team 
Conduct the Maneuver of a Squad 
Conduct Movement Techniques by a Platoon 
Conduct the Maneuver of a Platoon 


ILLUSTRATIONS: N/A 


TASK STEPS AND PERFORMANCE MEASURES: 

1. Soldiers immediately assume the nearest covered positions. 

2. Soldiers return fire immediately on reaching the covered positions. 

3. Squad/team leaders locate and engage known or suspected enemy positions with well- 
aimed fire, and pass information to the platoon/squad leader. 

4. Fire team leaders control the fire of their soldiers by using standard fire commands (initial 
and supplemental) containing the following elements: 

a. Alert. 

b. Direction. 

c. Description. 

d. Range. 

e. Method of fire (manipulation and rate of fire). 

f. Command to commence firing. 

5. Soldiers maintain contact (visual or oral) with the soldiers on their left or right. 

6. Soldiers maintain contact with the team leader and indicate the location of the enemy 
positions. 

7. The leaders (visually or orally) check the status of their personnel. 

8. The squad/fire team leaders maintain visual contact with the platoon/squad leader. 

9. The platoon/squad leader moves up to the squad/fire team in contact and links up with its 
leader. 

a. The platoon leader brings his RATELO, platoon FO, the squad leader of the nearest 
squad, and one machine gun team. 

b. The squad leader of the trail squad moves to the front of his lead fire team. 

c. The platoon sergeant moves forward with the second machine gun team and links up 
with the platoon leader, ready to assume control of the base-of-fire element. 

10. The platoon/squad leader determines whether or not his unit must move out of the 
engagement area. 

11. The platoon/squad leader determines whether or not his unit can gain and maintain 
suppressive fires with the element already in contact (based on the volume and accuracy 
of enemy fires against the element in contact). 
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TASK STEPS AND PERFORMANCE MEASURES: 

12. The platoon/squad leader makes an assessment of the situation. He identifies— 

a. The location of the enemy position and obstacles. 

b. The size of the enemy force engaging the unit in contact. (The number of enemy 
automatic weapons, the presence of any vehicles, and the employment of indirect 
fires are indicators of enemy strength.) 

c. Vulnerable flanks. 

d. Covered and concealed flanking routes to the enemy positions. 

13. The platoon/squad leader determines the next course of action (for example, fire and 
movement, assault, breach, knock out bunker, enter and clear a building or trench). 

14. The platoon/squad leader reports the situation to the company commander/platoon leader 
and begins to maneuver the unit. 

15. The platoon leader calls for and adjusts indirect fire (mortars or artillery), (squad leaders 
relay request through the platoon leader.) 

16. Leaders relay all commands and signals from the platoon chain of command. 

17. The platoon sergeant positions the BFVs to observe and to provide supporting fires. 

NOTE: Once the platoon has executed the React to Contact Drill, the platoon leader 
makes a quick assessment of the situation (for example, enemy size, location). He decides on a 
course of action. The platoon leader reports the situation to the company commander. 


SUPPORTED T&EO'S 


ARTEP NUMBER 

T&EO NUMBER 

T&EO TASK TITUE 

ARTEP 7-10-MTP 

07-2-1045 

Conduct a Defense (Infantry Company) 


07-2-1081 

Conduct a Link-up (Infantry Company) 


07-2-1090 

Conduct a Movement to Contact (Antiarmor/Infantry Company) 


07-2-1135 

Conduct a Raid (Infantry Company) 


07-2-1270 

Conduct an Infiltration or Exfiltration (Infantry Company) 


07-2-1279 

Conduct Convoy Escort (Antiarmor/Infantry Company) 


07-2-1315 

Conduct Patrol Operations (Infantry Company) 


07-2-1342 

Conduct Tactical Movement (Infantry Company) 


07-2-1369 

Cross a Water Obstacle (Dismounted) (Infantry Company) 


07-2-1468 

Take Action on Contact (Infantry Company) 


07-2-1477 

Breach an Obstacle (Infantry Company) 


07-2-1486 

Conduct Operations with Armored or Mechanized Infantry Vehicles in an 
Urban Environment (Infantry Company) 


07-2-2009 

Conduct a Route Reconnaissance (Infantry Company) 


07-2-2027 

Establish Observation Posts (Infantry Company) 

ARTEP 7-12-MTP 

07-2-1045 

Conduct a Defense (Infantry Company) 


07-2-1081 

Conduct a Link-up (Infantry Company) 


07-2-1090 

Conduct a Movement to Contact (Antiarmor/Infantry Company) 


07-2-1135 

Conduct a Raid (Infantry Company) 


07-2-1270 

Conduct an Infiltration or Exfiltration (Infantry Company) 


07-2-1279 

Conduct Convoy Escort (Antiarmor/Infantry Company) 


07-2-1315 

Conduct Patrol Operations (Infantry Company) 


07-2-1342 

Conduct Tactical Movement (Infantry Company) 


07-2-1369 

Cross a Water Obstacle (Dismounted) (Infantry Company) 
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__ SUPPORTED T&EO'S _ 

ARTEP NUMBER T&EO NUMBER T&EO TASK TITLE 

07-2-1468 Take Action on Contact (Infantry Company) 

_ 07-2-1477 _ Breach an Obstacle (Infantry Company) _ 

07-2-1486 Conduct Operations with Armored or Mechanized Infantry Vehicles in an 
Urban Environment (Infantry Company) 

07-2-2009 Conduct a Route Reconnaissance (Infantry Company) 

_ 07-2-2027 _ Establish Observation Posts (Infantry Company) _ 

ARTEP 7-4-MTP _ 07-3-1081 _ Conduct a Link-up (Infantry/Mortar/Reconnaissance Platoon/Squad) _ 

07-3-1090 Conduct a Movement to Contact (Infantry/Reconnaissance Platoon/Squad) 
07-3-1144 Conduct a Screen (Infantry/Reconnaissance Platoon/Squad) 

_ 07-3-1153 _ Conduct a Security Patrol (Infantry/Reconnaissance Platoon/Squad) _ 

_ 07-3-1189 _ Conduct Actions at Danger Areas (Infantry/Reconnaissance Platoon) _ 

07-3-1216 Conduct an Infiltration or Exfiltration (Infantry/Reconnaissance 

Platoon/Squad) 

07-3-1270 Conduct Tactical Movement (Mounted or Dismounted) 
(Antiarmor/Infantry/Mortar/Reconnaissance Platoon/Squad) 

07-3-1288 Cross a Water Obstacle (Dismounted) (Infantry/Reconnaissance 

_ Platoon/Squad) _ 

07-3-1432 Take Action on Contact (Infantry/Mortar/Reconnaissance Platoon/Squad) 
07-3-2000 Conduct a Route Reconnaissance (Infantry/Reconnaissance Platoon/Squad) 
ARTEP 7-5-MTP 07-3-1027 Breach an Obstacle (Infantry Platoon/Squad) 

_ 07-3-1054 _ Conduct a Defense (Infantry Platoon/Squad) _ 

07-3-1081 Conduct a Link-up (Infantry/Mortar/Reconnaissance Platoon/Squad) 

07-3-1090 Conduct a Movement to Contact (Infantry/Reconnaissance Platoon/Squad) 

_ 07-3-1126 _ Conduct a Raid (Infantry Platoon/Squad) _ 

_ 07-3-1144 _ Conduct a Screen (Infantry/Reconnaissance Platoon/Squad) _ 

07-3-1153 Conduct a Security Patrol (Infantry/Reconnaissance Platoon/Squad) 

07-3-1189 Conduct Actions at Danger Areas (Infantry/Reconnaissance Platoon) 

07-3-1216 Conduct an Infiltration or Exfiltration (Infantry/Reconnaissance 

_ Platoon/Squad) _ 

07-3-1225 Conduct Convoy Escort (Infantry Platoon/Squad) 

07-3-1243 Conduct Operations with Armored or Mechanized Vehicles in an Urban 

Environment (Infantry Platoon/Squad) 

07-3-1270 Conduct Tactical Movement (Mounted or Dismounted) 

_ (Antiarmor/Infantry/Mortar/Reconnaissance Platoon/Squad) _ 

07-3-1288 Cross a Water Obstacle (Dismounted) (Infantry/Reconnaissance 

_ Platoon/Squad) _ 

07-3-1432 Take Action on Contact (Infantry/Mortar/Reconnaissance Platoon/Squad) 

07-3-2000 Conduct a Route Reconnaissance (Infantry/Reconnaissance Platoon/Squad) 

ARTEP 7-7J-MTP _ 07-3-1027 _ Breach an Obstacle (Infantry Platoon/Squad) _ 

07-3-1054 Conduct a Defense (Infantry Platoon/Squad) 

07-3-1081 Conduct a Link-up (Infantry/Mortar/Reconnaissance Platoon/Squad) 

07-3-1090 Conduct a Movement to Contact (Infantry/Reconnaissance Platoon/Squad) 

_ 07-3-1126 _ Conduct a Raid (Infantry Platoon/Squad) _ 

07-3-1144 Conduct a Screen (Infantry/Reconnaissance Platoon/Squad) 

07-3-1153 Conduct a Security Patrol (Infantry/Reconnaissance Platoon/Squad) 

_ 07-3-1189 _ Conduct Actions at Danger Areas (Infantry/Reconnaissance Platoon) _ 

07-3-1216 Conduct an Infiltration or Exfiltration (Infantry/Reconnaissance 

_ Platoon/Squad) _ 

07-3-1225 Conduct Convoy Escort (Infantry Platoon/Squad) 

07-3-1243 Conduct Operations with Armored or Mechanized Vehicles in an Urban 

Environment (Infantry Platoon/Squad) 

07-3-1270 Conduct Tactical Movement (Mounted or Dismounted) 

_ (Antiarmor/Infantry/Mortar/Reconnaissance Platoon/Squad) _ 

07-3-1288 Cross a Water Obstacle (Dismounted) (Infantry/Reconnaissance 

Platoon/Squad) 
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__ SUPPORTED T&EO'S _ 

ARTEP NUMBER T&EO NUMBER T&EO TASK TITLE 

07-3-1432 Take Action on Contact (Infantry/Mortar/Reconnaissance Platoon/Squad) 

_ 07-3-2000 _ Conduct a Route Reconnaissance (Infantry/Reconnaissance Platoon/Squad) 

ARTEP 7-8-MTP _ 07-3-1027 _ Breach an Obstacle (Infantry Platoon/Squad) _ 

_ 07-3-1054 _ Conduct a Defense (Infantry Platoon/Squad) _ 

07-3-1081 Conduct a Link-up (Infantry/Mortar/Reconnaissance Platoon/Squad) 

_ 07-3-1090 _ Conduct a Movement to Contact (Infantry/Reconnaissance Platoon/Squad) 

_ 07-3-1126 _ Conduct a Raid (Infantry Platoon/Squad) _ 

07-3-1144 Conduct a Screen (Infantry/Reconnaissance Platoon/Squad) 

07-3-1153 Conduct a Security Patrol (Infantry/Reconnaissance Platoon/Squad) 

07-3-1171 Conduct a Tactical Road March (Dismounted) (Infantry/Reconnaissance 

_ Platoon/Squad) _ 

_ 07-3-1189 _ Conduct Actions at Danger Areas (Infantry/Reconnaissance Platoon) _ 

07-3-1216 Conduct an Infiltration or Exfiltration (Infantry/Reconnaissance 

Platoon/Squad) 

07-3-1243 Conduct Operations with Armored or Mechanized Vehicles in an Urban 

_ Environment (Infantry Platoon/Squad) _ 

07-3-1270 Conduct Tactical Movement (Mounted or Dismounted) 
(Antiarmor/Infantry/Mortar/Reconnaissance Platoon/Squad) 

07-3-1288 Cross a Water Obstacle (Dismounted) (Infantry/Reconnaissance 

Platoon/Squad) 

_ 07-3-1432 _ Take Action on Contact (Infantry/Mortar/Reconnaissance Platoon/Squad) _ 

07-3-2000 Conduct a Route Reconnaissance (Infantry/Reconnaissance Platoon/Squad) 

ARTEP 7-90-MTP 07-3-1081 Conduct a Link-up (Infantry/Mortar/Reconnaissance Platoon/Squad) 

07-3-1270 Conduct Tactical Movement (Mounted or Dismounted) 

_ (Antiarmor/Infantry/Mortar/Reconnaissance Platoon/Squad) _ 

_ 07-3-1432 _ Take Action on Contact (Infantry/Mortar/Reconnaissance Platoon/Squad) _ 

ARTEP 7-91-MTP 07-2-1036 Conduct a Defense (Antiarmor Company/Platoon) 

07-2-1090 Conduct a Movement to Contact (Antiarmor/Infantry Company) 

_ 07-2-1279 _ Conduct Convoy Escort (Antiarmor/Infantry Company) _ 

_ 07-2-1459 _ Take Action on Contact (Antiarmor Company/Platoon) _ 

07-3-1270 Conduct Tactical Movement (Mounted or Dismounted) 

(Antiarmor/Infantry/Mortar/Reconnaissance Platoon/Squad) 

ARTEP 7-92-MTP _ 07-3-1081 _ Conduct a Link-up (Infantry/Mortar/Reconnaissance Platoon/Squad) _ 

_ 07-3-1090 _ Conduct a Movement to Contact (Infantry/Reconnaissance Platoon/Squad) 

_ 07-3-1144 _ Conduct a Screen (Infantry/Reconnaissance Platoon/Squad) _ 

07-3-1153 Conduct a Security Patrol (Infantry/Reconnaissance Platoon/Squad) 

07-3-1171 Conduct a Tactical Road March (Dismounted) (Infantry/Reconnaissance 

Platoon/Squad) 

_ 07-3-1189 _ Conduct Actions at Danger Areas (Infantry/Reconnaissance Platoon) _ 

07-3-1216 Conduct an Infiltration or Exfiltration (Infantry/Reconnaissance 

Platoon/Squad) 

07-3-1270 Conduct Tactical Movement (Mounted or Dismounted) 

(Antiarmor/Infantry/Mortar/Reconnaissance Platoon/Squad) 

07-3-1288 Cross a Water Obstacle (Dismounted) (Infantry/Reconnaissance 

_ Platoon/Squad) _ 

07-3-1432 Take Action on Contact (Infantry/Mortar/Reconnaissance Platoon/Squad) 

07-3-2000 Conduct a Route Reconnaissance (Infantry/Reconnaissance Platoon/Squad) 

ARTEP 7-93-MTP 07-5-1001 Conduct Surveillance (LRS Team) 

_ 07-5-1002 _ Reconnoiter Area (LRS) _ 

07-5-1003 Reconnoiter Zone (LRS) 

07-5-1004 Assess Damage 

_ 07-5-1101 _ Conduct Airborne Insertion _ 

_ 07-5-1102 _ Conduct Helicopter Insertion/Extraction _ 

07-5-1103 Conduct Ground Infiltration/Exfiltration 

07-5-1107 Move Tactically (LRS) 
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SUPPORTED T&EO'S 


ARTEP NUMBER 

T&EO NUMBER 

T&EO TASK TITUE 


07-5-1108 

Cross Danger Area 


07-5-1109 

Cross Water Obstacle (LRS) 


07-5-1110 

Establish Hide Site 


07-5-1111 

Establish Surveillance Site 


07-5-1112 

Conduct Linkup (LRS Team) 


07-5-1115 

Establish a Patrol Base 


07-5-1201 

Acquire a Target 


07-5-1401 

Evade and Recover 


07-5-1406 

React to Indirect Fire (LRS) 


07-5-1502 

Establish/Recover a Cache 


07-5-1605 

Consolidate and Reorganize (LRS) 
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APPENDIX D. INSTITUTIONAL REVIEW BOARD DOCUMENTS 


A. MEMORANDUM REQUESTING IRB APPROVAL 



Jason M. Jones, Major, U.S. Army 
Joseph M. Nolan, Major, U.S. Army 
Naval Postgraduate School 
Monterey, California 93943 


831-656-3067 

jmjones@nps.edu 

jmnolan@nps.edu 


To: Protection of Human Subjects Committee 

Subject: Application for Human Subjects Review for Games for Training: Examining the 

Use of Multiplayer, First-Person Perspective Commercial Games to Train Infantry 

Squads 

1. Attached is a set of documents outlining a proposed experiment to be conducted over 
the next eight months for our thesis project. 

2. We are requesting approval of the described experimental protocol. An experimental 
outline is included for your reference that describes the methods and measures we 
plan to use. 

3. We include the consent forms, privacy act statements, all materials and forms that a 
subject will read or fill-out, and the debriefing forms (if applicable) we will be using 
in the experiment. 

4. We understand that any modifications to the protocol or instruments/measures will 
require submission of updated IRB paperwork and possible re-review. Similarly, we 
understand that any untoward event or injury that involves a research participant will 
be reported immediately to the IRB Chair and NPS Dean of Research. 


Jason M. Jones 


Joseph M. Nolan 
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B. APPLICATION FOR IRB APPROVAL 


APPLICATION POR 

HUMAN SUBJECTS REVIEW (HSR) 

HSR NUMBER (to be assigned) 

PRINCIPAL INVESTIGATOR(S) (Full Name, Code, Telephone) 

Jason M. Jones, MAJ, U.S. Army, (831) 277-9582 

Joseph M. Nolan, MAJ, U.S. Army, (831) 901-5462 

APPROVAL REQUESTED [ X ] New 

[ 

] Renewal 

LEVEL OP RISK [ ] Exempt [ X ] Minimal 
Justification: 

[ ] More than Minimal 

WORK WILL BE DONE IN (Site/Bldg/Rm) 
Classroom XXI, Port Benning, GA 

ESTIMATED NUMBER OP DAYS TO 
COMPLETE 

10 Days 

MAXIMUM NUMBER OP SUBJECTS 

40 Subjects 

ESTIMATED LENGTH OP EACH 
SUBJECT’S PARTICIPATION 

12 Hours 


SPECIAL POPULATIONS THAT WILL BE USED AS SUBJECTS 

[ ] Subordinates [ ] Minors [ ] NPS Students [ ] Special Needs (e.g. Pregnant women) 


Specify safeguards to avoid undue influence and protect subject’s rights: 


OUTSIDE COOPERATING INVESTIGATORS AND AGENCIES 
U.S. Infantry Officer Basic Course, Eort Benning, GA 31905 

[ ] A copy of the cooperating institution’s HSR decision is attached. 


TITLE OE EXPERIMENT AND DESCRIPTION OP RESEARCH (attach additional sheet if needed). 
Games for Training: Examining the Use of Multiplayer, Pirst-Person Perspective Commercial Games to Train 
Infantry Squads. Using squads of students from the Infantry Officer Basic Course we will measure the 
effectiveness of using networked games to train dismounted tactics. 


I have read and understand NPS Notice on the Protection of Human Subjects. If there are any changes in 
any of the above information or any changes to the attached Protocol, Consent Porm, or Debriefing Statement, I 
will suspend the experiment until I obtain new Committee approval. 

SIGNATURE_ DATE_ 
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c. 


PARTICIPANT CONSENT FORM 


1. Introduction. You are invited to participate in a study on the use of commercially 
available simulations to train dismounted tactics at the squad level. With information 
gathered from you and other participants, we want to see how effective training in the 
virtual environment is compared to traditional instruction. We ask you to read and sign 
this form indicating that you agree to be in the study. Please ask any questions you 
may have before signing. 

2. Background Information. We are conducting this study as part of our master’s 
degree thesis at the Naval Postgraduate School. 

3. Procedures. If you agree to participate in this study, the researcher will explain the 
tasks in detail. There will be two sessions: 1) simulation fa mi liarization phase and 2) 
mission execution phase. Each will last approximately five hours. During these phases 
you will execute individual and squad actions at a computer work station. 

4. Risks and Benefits. This research involves no risks or discomforts greater then those 
encountered when using a computer keyboard and mouse. The benefits to the 
participants are improving your knowledge of tactics, techniques and procedures for 
tactical operations and contributing to research on the effectiveness of training 
techniques. 

5. Compensation. No tangible reward will be given. A copy of the results will be 
available to you at the conclusion of the experiment. 

6. Confidentiality. The records of this study will be kept confidential. No information 
will be publicly accessible which could identify you as a participant. 

7. Voluntary Nature of the Study. If you agree to participate, you are free to withdraw 
from the study at any time without prejudice. You will be provided a copy of this form 
for your records. 

8. Points of Contact. If you have any further questions or comments after the completion 
of the study, you may contact the research supervisor. Dr. Rudolph P. Darken (831) 
656-4072 darken@nps.navy.mil. 

9. Statement of Consent. I have read the above information. I have asked all questions 
and have had my questions answered. I agree to participate in this study. 


Participant’s Signature 


Date 


Date 


Researcher’s Signature 
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D. MINIMAL RISK CONSENT STATEMENT 


MINIMAL RISK CONSENT STATEMENT 

NAVAL POSTGRADUATE SCHOOL, MONTEREY, CA 93943 

Participant: VOLUNTARY CONSENT TO BE A RESEARCH PARTICIPANT 

IN: Games for Training: Examining the Use of Multiplayer, Eirst-Person Perspective 

Commercial Games to Train Infantry Squads 

1. I have read, understand and been provided "Information for Participants" that provides the 
details of the below acknowledgments. 

2. I understand that this project involves research. An explanation of the purposes of the 
research, a description of procedures to be used, identification of experimental procedures, 
and the extended duration of my participation have been provided to me. 

3. I understand that this project does not involve more than minimal risk. I have been informed 
of any reasonably foreseeable risks or discomforts to me. 

4. I have been informed of any benefits to me or to others that may reasonably be expected from 
the research. 

5. I have signed a statement describing the extent to which confidentiality of records identifying 
me will be maintained. 

6. I have been informed of any compensation and/or medical treatments available if injury 
occurs and is so, what they consist of, or where further information may be obtained. 

7. I understand that my participation in this project is voluntary, refusal to participate will 
involve no penalty or loss of benefits to which I am otherwise entitled. I also understand that 
I may discontinue participation at any time without penalty or loss of benefits to which I am 
otherwise entitled. 

8. I understand that the individual to contact should I need answers to pertinent questions about 
the research is Professor Rudy Darken, Principal Investigator, and about my rights as a 
research participant or concerning a research related injury is Prof. Jim Eagle, Operations 
Research Dept. Chairman. A full and responsive discussion of the elements of this project 
and my consent has taken place. NPS Medical Monitor: CAPT Nick Davenport, MC, USN, 
Flight Surgeon, Naval Postgraduate School (831) 656-7876, nadavenp@nps.navy.mil 


Signature of Principal Investigator Date 


Signature of Volunteer 


Date 
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E. PRIVACY ACT STATEMENT 


PRIVACY ACT STATMENT 

NAVAL POSTGRADUATE SCHOOL, MONTEREY, CA 93943 
PRIVACY ACT STATEMENT 


1. Purpose: Data on my opinions, observations and performance will be collected to 
enhance knowledge, and to develop simulations, procedures, and equipment to 
improve the development of Virtual Environments. 

2. Use: My opinions, observations and performance data will be used for statistical 
analysis by the Departments of the Navy and Defense, and other U.S. Government 
agencies, provided this use is compatible with the purpose for which the information 
was collected. Use of the information may be granted to legitimate non-government 
agencies or individuals by the Naval Postgraduate School in accordance with the 
provisions of the Ereedom of Information Act. 

3. Disclosure/Confidentiality: 

a. I have been assured that my privacy will be safeguarded. I will be assigned a 
control or code number which thereafter will be the only identifying entry on 
any of the research records. The Principal Investigator will maintain the cross- 
reference between name and control number. It will be decoded only when 
beneficial to me or if some circumstances, which is not apparent at this time, 
would make it clear that decoding would enhance the value of the research data. 
In all cases, the provisions of the Privacy Act Statement will be honored. 

b. I understand that a record of the information contained in this Consent Statement 
or derived from the experiment described herein will be retained permanently at 
the Naval Postgraduate School or by higher authority. I voluntarily agree to its 
disclosure to agencies or individuals indicated in paragraph 3 and I have been 
informed that failure to agree to such disclosure may negate the purpose for 
which the experiment was conducted. 

c. I also understand that disclosure of the requested information, including my 
Social Security Number, is voluntary. 

_ Name, 

Grade/Rank (if applicable) DOB SSN 

[Please print] 


Signature of Volunteer 
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APPENDIX E. EXPERIMENT SET UP USING THE MISSION 

EDITOR 

A. OVERVIEW 

This section describes the use of the Delta Force: Black Flawk Down - Team 
Sabre™ MED that is included with this COTS game. The mission editor (MED) is a 
GUI based tool that allows the user to create robust virtual environments for game play 
on the Delta Force: Black Flawk Down - Team Sabre™ platform. The software includes 
a MED manual as an Adobe'^’^ AcrobaUM document, and the manual includes a basic 
tutorial for learning the GUI and creating basic game levels. Although we will defer to 
the MED for specifics on how to create a game level, we will present screen captured 
images and basic details for the creation of the eight lanes we created for our experiment. 
A full copy of the MED manual can be found at the following web site (last accessed 
September 2005): 

http://www.dfb arracks.com/downloads/index.php?action=df&fid=217 

B. GAME MISSION AREAS 

In the following section, we will provide a brief description of each event area, 
key events, and any other pertinent data required to recreate this experiment. 

I. Mission One Area 

The first four missions we created replicated a day time desert environment that 
covered approximately three square miles of virtual terrain and was based on the current 
contemporary operating environment. The starting point for the mission was a forward 
operating base located two miles north of a prominent east-west river and five miles 
outside a notional local city (Eigure 9). The forward operating base included an airplane 
hangar, maneuver and live fire lanes for subjects to practice movement and weapon 
coordination, and a UH-60 helicopter for insertion into the objective areas. Additional 
items included a wooden platform for the group to use when rehearsing entering the 
helicopter, a truck (programmed to drive to the objective area if the helicopter failed), and 
other associated models to create a look and feel of the forward operating base (Eigure 
10 ). 
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Figure 9. Mission One Area 
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Figure 10. Forward operating Base: Mission One Area 
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The first objective in the Mission One Area was a radio tower located along the 
northern edge of the east-west river in sector. Seven enemy personnel were placed on the 
objective with a combination of AK-47 rifles and rocket propelled grenade (RPG) 
launchers. The accuracy of the AI for the enemy agents was set to 60%, and all agents 
engaged from a fixed location (meaning no agents maneuvered against the subjects) 
(Figure 11). We added a fence line adjacent to the radio tower to replicate a hasty 
defensive perimeter, and added a truck behind the radio tower to as an enemy escape 
method. Finally, we added a spawn point on the objective to allow subjects killed during 
the exercise to quickly link up with their squad, and resupply items for players to refit 
after seizing the objective. 
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Figure 11. First Objective: Mission One Area 
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The second objective in the Mission One Area was an oil refinery located along 
the southern edge of the east-west river in sector. Seven enemy personnel were placed on 
the objective with a combination of AK-47 rifles and RPG launchers. The accuracy of 
the AI for the enemy agents was set to 60%, and all agents engaged from a fixed location 
(Figure 12). We added a bunker on a hill top near the oil pumps to provide overwatch on 
the objective, and to provide a scenario where our subjects would have to As before, we 
added a spawn point on the objective to allow subjects killed during the exercise to 
quickly link up with their squad, and resupply items for players to refit after seizing the 
objective. 



Figure 12. Second Objective: Mission One Area 
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The third objective in the Mission One Area was an enemy river boat launch site 
located along the southern edge of the east-west river in sector. Seven enemy personnel 
were placed on the objective with a combination of AK-47 rifles, .50 caliber machine 
guns and RPG launchers. The accuracy of the AI for the enemy agents was set to 60%, 
and all agents engaged from a fixed location (Figure 13). On this objective, three of the 
agents were placed in a defile (along the river) that forced the maneuvering squad to use 
grenade and area weapons fire to set conditions for success. On this objective, we placed 
the spawn point and resupply items on the opposite bank of the river in order to provide 
an opportunity for the squad to exercise methods for clearing a linear danger area as the 
unit crossed the river on the adjacent bridge following mission completion. 
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Figure 13. Third Objective: Mission One Area 


The forth (and final) objective in the Mission One Area was an enemy river boat 
launch site with a watchtower manned with a .50 caliber machine gun located along the 
northern edge of the east-west river in sector. Six enemy personnel were placed on the 
objective with a combination of AK-47 rifles, RPG launchers as well as the machine gun 
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position. The accuracy of the AI for the enemy agents was set to 70%, and all agents 
engaged from a fixed location (Figure 14). On this objective, three of the agents were 
placed in a defile (along the river) that forced the maneuvering squad to use grenade and 
area weapons fire to set conditions for success. We placed the tower in the defile so that 
the top of the tower (with manned machine gun position) crested the ridgeline in order for 
the enemy agent to have effective grazing fires against the maneuvering squad. Finally, 
we kept the vegetation near the objective to minimum in order to force the maneuver 
squad to think creatively about their approach to the site. 



Figure 14. Fourth Objective : Mission One Area 
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2. Mission Area Two 

The next four missions we created replicated a day time ocean coastal 
environment that covered approximately three square miles of virtual terrain and was 
based on the current contemporary operating environment. The starting point for the 
mission was a populated town located approximately two miles north of the objective 
areas (Figure 15). The town was populated with eleven civilian agents and four friendly 
Soldier agents to replicate a civil affairs team deployed in support of SASO operations 
(Figure 16). In the scenario, the residents of the coastal town provided support to the host 
nation’s off shore oil platform. To create this environment, we added boat docks and an 
off shore refinery that the maneuver squad would fly past during the helicopter insertion 
into the objective areas. The intent for this environment was to create a sense of 
operating in a civilian populated area where squad members had to use caution when 
using lethal force. 



Figure 15. Mission Two Area 
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Figure 16. Civilian Town: Mission Two Area 
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The first objective in the Mission Two Area was a civilian auto repair shop 
located along the western coast. Six enemy personnel were placed on the objective with 
a combination of AK-47 rifles, .50 caliber machine guns and RPG launchers. The 
accuracy of the AI for the enemy agents was set to 75%, and all agents engaged from a 
fixed location (Figure 17). We added two machine gun positions on this objective with 
the intent of covering both avenues of approach: the ocean coast and the north-south dirt 
road. Finally, we added a spawn point and resupply items inside the auto repair garage. 



Figure 17. First Objective: Mission Two Area 
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The second objective in the Mission Two Area was an enemy safe-house located 
east of the auto repair shop. Seven enemy personnel were placed on the objective with a 
combination of AK-47 rifles, .50 caliber machine guns and RPG launchers. Three of the 
seven agents were placed in a counter attack position and were triggered to engage after 
the squad seized the objective. The counter attack force was placed in this scenario in 
order to exercise the squad’s effectiveness to pull all around security after seizing the 
objective. The accuracy of the AI for the enemy agents on the objective was set to 75%, 
and the counter attack force was set to 100% (Figure 18). As before, we placed a spawn 
point and resupply items near the objective area. 



Figure 18. Second Objective: Mission Two Area 
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The third objective in the Mission Two Area was an enemy vehicle resupply point 
located south of the safe-house. Seven enemy personnel were placed on the objective 
with a combination of AK-47 rifles, and RPG launchers. The challenge of this objective 
was the terrain in which the maneuver squad had to negotiate to attack the enemy. The 
hill masses forced the squad to visual contact in order to maneuver. By forcing the squad 
leader to lose visual contact with his base of fire element, we hoped to exercise command 
and control, as well as communication tasks at all levels. The accuracy of the AI for the 
enemy agents on the objective was set to 75% (Figure 19). The spawn point and resupply 
items were placed west of the objective. 
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Figure 19. Third Objective: Mission Two Area 
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The last objective in the Mission Two Area was an enemy river boat launch site 
located along the western. Nine enemy personnel were placed on the objective with a 
combination of AK-47 rifles, .50 caliber machine guns and multiple RPG launchers 
providing overlapping fires of the main avenue of approach. The accuracy of the AI for 
the enemy agents was set to 80%, and all agents engaged from a fixed location (Figure 
20). On this objective, four of the agents were placed in a defile (along the river) that 
forced the maneuvering squad to use grenades, smoke and area weapons fire to set 
conditions for success. 



Figure 20. Fourth Objective: Mission Two Area 
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APPENDIX F. COGNITIVE TASK ANALYSIS 


A. OVERVIEW 

We conducted this task analysis to identify the human abilities required to 
conduct the squad “React to Contact” battle drill and map those human abilities to the 
COTS game Delta Force: Black Flawk Down - Team Sabre™ by Novalogic™. For the 
task analysis, we conducted a small scale experiment executing cooperative missions in 
multiplayer mode to determine the skill requirements for the COTS game and those of the 
battle drill. We found that potential training value exists to train infantry collective tasks, 
but the degree differs between cognitive leader tasks and individual Soldier skills. We 
determined that the networked COTS game cannot be used to train individual skills, but 
that the COTS game can be used as a training venue for unit collective training. 

B. TASK ANALYSIS OF “REACT TO CONTACT” 

In an effort to standardize our task analysis, we used the Occupational 
Information Network’s 0*NET Descriptors of human abilities (Occupational Information 
Network, 2005). Using these descriptors we thoroughly examined each performance 
measure of the squad “React to Contact” battle drill outlined in ARTEP 7-8 Drill to 
determine which human abilities were required to accomplish the task. Because our 
hypothesis concerned using the COTS game as a collective training tool, we selected only 
those performance measures unique to the collective task and did not focus effort 
analyzing in detail any individual skills. Nested individual tasks such as “Eoad, Unload 
and Reduce Stoppage on an M16A2 Rifle” that support collective tasks are required to be 
trained prior to conducting any collective training event. By making this assumption, we 
could feel confident that the trainees would not incur a negative training transfer by 
conducting tasks in a virtual environment that differed from reality. 

Not to discount individual skills entirely, we planned to use a participant survey 
during our experiment to get feedback from the users to determine the ability of the 
COTS game to replicate sensory cues required to perform individual skills that we did not 
address with 0*NET Descriptors. Doing so allowed us to focus on cognitive leader 
performance measures, which are often difficult to train without dedicated resources. 


149 



using the 0*NET to determine if the required human abilities can be mapped to a COTS 
game platform. Based on our assessment, we determined the following human abilities 
required to conduct selected squad “React to Contact” performance measures (Table 24): 
Table 24. “React to Contact” Task Analysis using 0*NET Descriptors 


1. Soldiers iinmcdiatclv assume the nearest covered positions. (Individual Task - Surv'ev) 

2. Soldiers return fire iinmcdialclv on reachms the covered positions. (Individual Task - Survev) 

Squad team leaders locate and engage known or suspected cnemv positions with wclEatmcd (ire. and pass infomiaiion to the 

platoon/SQuad leader. 




• (Dnil Expression 

• Visualization 

• Far Vision 

• Sound Localization 

♦ Indcduciivc Reasoning 

• Selective Aiicniion 

• Pcri{riicrial Vision 

• Near Vision 

• Information Ordering 

• Rate Control 

• Depth Perception 

• Spatial Orientation 

♦ Flcxibililv of Closcurc 

• Arm - Hand Steadiness 

• Hearing Sensitivity 


4. Fire team leaders control the fire of their soldiers by using standard fire commands (initial and supplemental) containing the following 

elements: 




4a- Alert. 

• Oral Expression 

• Speech Clarity 



4b. Dirccticwi. 

• Oral Expression 

• Speech Clarity 

• Fluency of Ideas 


4c. Description. 1 

• Oral Expression 

• Information Ordering 



• Speech Clarity 

• Flexibility of Closure 



4d. Range. 

• Oral Expression 

• Depth Perception 

• Information Ordenng 

• Spatial Orientation 

• Speech Clarity 




4e. Method of (Ire (manipulation and rate of fire). 

• Oai Expression 

• Riitc Control 

• Speech Clarity 


4f. Command to commence firing 

• Oral Expression 

• Speech Clarity 



5. Soldiers mainiam contact (visual or oral) with the soldiers on their led or right. (Individual Task - Surv'CV) 

6. Soldiers maintain contact with the team leader and indicate the location of tltc enemy positions. (Individual Task Survev) 

7. The lenders (visually or orally) check the status of their personnel. 

• (Dnil Expression 

• Infbnnalion Ordering 

• Spatial Oriaitalion 

• Category Flexibility 

• Speech Clarnv 

• Depth PerceptKMi 

• Inductive Reasoning 


K. The SQuad/ilrc team Icsiders maintain visual contact with the platoon/sauad leader. 

• Spatial Orientation 

• Near Vision 

• Depth PcTccpiiou 

• Sound Localization 

• Visualization 

• Far Vision 

• Hciiring Scnsiliviiy 


• Selective Attention 

• Peripherial Vision 



*}. The platoon/SQuad leader moves up to the sQuad/(irc lean 

in contact and links up with its leader. (Not Evaluated) 

l)a. The platoon leader brings his RATELO. platoon FO. the sauad leader of the nearest sauad, and one machine gun team. 

*>b. The sauad leader of the trail sauad n>ovcs to the front of his Ictid fire team. 

9c. Tltc platoon sergeant moves forward with the second madtinc gun team and links up with the platoon leader, ready to assume 

control of the basc*of-firc ctcmcni. 



10. The plaiooci/sauad leader dctcnuincs whether or not his unit must move out of the engagement area. 

• Indcductivc Reasoning 

• Visualization 

• Depth Perception 

• Spatial Orientation 

• Information Ordering 

• Selective Aiicniion 

• Hearing SensHivitv 

• Peripherial Vision 

11. The pintoon s<|uad leader determines whether or not his unit can gain and maintain siqtprcssive fires with the dement already in 

contact (based on the volume and accuracy of enemy fires against the element in contact). 


• Indcductivc Reasoning 

• Spatial Orientation 

• Far Vision 

• Sound Localuaiion 

• Information Ordering 

• Visualization 

• Peripherial Vision 

• Hearing Sensitivity 

• Category Flexibility 

• Selective Attention 

• Depth Perception 

• Near V'ision 

• Flexibility of Closeure 




12. The platoon/sauad laidcr makes an assessment of the siliution. He idciilifics— 

12a. The location of the enemy position and obstacles. 

• Spatial Oricniaiion 

• Near Vision 

• Depth Perception 

• Sound Localization 

• Visualization 

• Far Vision 

• Hciiring Scnsiliviiy 

• Penpherial Vision 

♦ Selective Attention 




12b. The size of the enemy force engaging the unit in contact. (The number of cnany automatic weapons, the presence of any 

vehicles, and the employment of indirect fires arc indicators of enemy strength.) 


• Indcduciivc Reasoning 

• Near Vision 

• Depth PcrceptitMi 

• Sound Localization 

• Informatiou Ordering 

• Far V'ision 

• Hearing Sensitivity 

• Penpherial Vision 

• Speed of Closcurc 




12c. Vulnerable flanks. 




• Indcductivc Reasoning 

• Near V'ision 

• Peripherial V’ision 

• Depth f^rcepiion 

« Information Ordering 

• Far Vision 



12d. Covered and concealed flanking routes to the enemy positions. 

• Indcductivc Reasoning 

• Near Vision 

• Peripherial Vision 

• Depth Perception 

« Information Ordering 

• Far Vision 
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Our next step was selection of a COTS game platform (See Chapter III for 
selection criteria). The game we chose was Delta Force: Black Flawk Down - Team 
Sabre™ by Novalogic®. This game is a first person shooter game with multiplayer 
capability that is controlled using a keyboard and mouse configuration. The video 
display screen is broken into three main areas: environment image, HUD, and messages. 
The environment image displays the environment in the direction you are facing, a cross 
hair for aiming your weapon, your “hands” and an image of the weapon you currently 
have selected. The HUD features a compass, waypoint indicator, an image of the 
selected weapon, ammunition remaining, and an image representing your current posture 
(prone, crouching, or standing). The message section displays the last three text 
messages the player received and their character name. 

We conducted a limited experiment using two collaborative mission scenarios 
created using the Delta Force: Black Flawk Down - Team Sabre™ MED. We fought the 
both scenarios several times under different conditions to determine the optimal training 
setting. For tactical orders preparation, we pulled the intelligence scenario, maps and 
friendly situation from the MED. Using this information, we constructed a map of the 
entire battle-space, defined our area of operations, and went through a condensed orders 
process. We produced a set of enemy and friendly graphics with a fragmentary tactical 
order for use in the scenario. Upon completion of a rehearsal run (to become familiar 
with the controls and key mapping), we conducted our final iterations and used our 
“React to Contact” performance measures with the human abilities table to capture our 
results. We based our assessment on the game’s ability to provide sensory cues to support 
specific physical human abilities and how the structured scenario we developed provided 
other cues to support cognitive human abilities. For example, we assessed the ability of 
the game to support “Peripheral Vision”, and the scenario’s ability to provide enough 
cues to support “Information Ordering”. We used a five point Eikert scale as our rating 
to map the human abilities to the COTS game platform with 1 being a low or poor 
matching and a 5 being high or a very good match (Table 25). 
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Table 25. Human Abilities Assessment of COTS Game 


Rating Scale 

0*NET Human Ability 

Very Low 

Low 

Average 

High 

Very High 

Arm - Hand Steadiness 

X 





Depth Perception 


X 




Far Vision 

X 





Hearing Sensitivity 



X 



Near Vision 




X 


Peripheral Vision 

X 





Sound Localization 



X 



Spatial Orientation 



X 



Category Flexibility 




X 


Deductive Reasoning 




X 


Flexibility of Closure 




X 


Fluency of Ideas 




X 


Inductive Reasoning 




X 


Information Ordering 




X 


Oral Comprehension 




X 


Oral Expression 




X 


Rate Control 



X 



Selective Attention 




X 


Speech Clarity 




X 


Speed of Closure 




X 


Visualization 




X 



C. COTS GAME ANALYSIS USING ELEMENTS OF COMBAT POWER 

In addition to the human abilities defined by 0*NET Descriptors, we examined 
elements of combat power that are nested within the “React to Contact” battle drill and 
how they could be trained with a COTS game in a structured training scenario. We 
defined these elements in the following categories: 

• Unit Specific Standard Operating Procedures 

• Tactical Reporting 

• Tactical Movement 

• Small Unit Fire Control and Distribution and Direct Fire Planning 

• Intelligence Preparation of the Battlefield 

• Command and Control 

By looking further that the performance measures and human abilities, we were 
able to develop a broader picture of the ability of Delta Force: Black Hawk Down - Team 
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Sabre™ to be used as a networked training platform. We defined these elements of 
eombat power as it related to the COTS game trainer in the following manner: 

• Unit Speeifie Standard Operating Proeedures (SOPs): The training system 
must be able to support a variety of small unit SOPs. From teehniques for 
erossing a linear danger area to proeedures for entering a building and elearing a 
room, the trainer must be able to support training these skills by allowing every 
member of the unit to have an avatar in the virtual environment. With every 
member of the unit in the virtual environment (VE), the whole team ean see and 
hear the task performed. Every member of the unit can also participate in the 
procedure and place himself in relation to his team members, the enemy and the 
environment. The system must also support different conditions to allow for 
training in all types of environments. 

• Tactical Reporting: The training system must be able to provide sufficient 
cues for leaders to send detailed tactical reports to the higher headquarters. The 
more robust the virtual environment, the better it will be to train subordinate 
leaders on the ability to pick out the critical information required and filter the 
sensory static. 

• Tactical Movement: The system must be able to reinforce basic tactical 
movement techniques. By placing the individual users in the VE with assigned 
avatars, the trainer must be able to provide enough sensory cues to allow small 
unit tactical movement. The cues must provide the individual user with sufficient 
signals to let him know where he is in relation to his flank team member, squad 
leader and platoon leader. The goal for reinforcing this task is to ensure proper 
spacing while moving through a series of different environments and under 
differing conditions. 

• Small Unit Eire Control and Distribution and Direct Eire Planning (DPP): 
The training system must provide a fluid event scenario with a scalable enemy 
force in order to properly train a small units’ ability to control subordinate rates of 
weapon fire and the distribution of those fires in a real-time tactical environment. 
The system should also provide a ground tactical scenario and maps in order to 
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allow for leader orders process, and generation of graphic control measures that 
support his direct fire plan. 

• Intelligence Preparation of the Battlefield (IPB): The tactical scenario 
from the training system must also provide enough of an intelligence picture, in 
addition to the aforementioned maps, to allow the leader to generate his own 
intelligence picture of his sector for use in developing the ground tactical plan. 

• Command and Control: The system needs to allow for the small unit 
leadership the ability to maneuver his forces in a virtual environment as he would 
on real terrain. The platoon/squad leader’s ability to direct subordinate unit 
leaders movements in the VE in accordance with unit SOPs and the ground 
tactical plan is critical for training control. Communications must be replicated 
(or augmented with unit assigned equipment, for example: AN/PRC-119 
SINCGARS radios) in the VE to force proper radio procedures and discourage the 
artificiality of calling a subordinate leader from one workstation to another sans 
radio. 

We found that the true training value for an infantry squads lies with the 
leadership in the squad (squad and team leaders) and only somewhat with the individual 
Soldier. The squad leadership in the game scenario is faced with decisions they are likely 
to make in a combat scenario: “Where are enemy forces?”, “Which route is best?”, “I see 
the enemy, should I engage now or wait?”, “Now that we’re in contact, what should we 
do?” All of these questions draw on the leader’s cognitive ability to recognize and react 
to a given situation. 

Without guidance and enforcing use of unit standards by leaders, it is unlikely that 
an individual soldier would walk away from a session better trained. The individual 
abilities within the game do not match well to the performance of actual dismounted 
tactics (i.e. there is no left-mouse button to push on my weapon), but within the context 
of collective training, there is an assumption that individual soldiers are trained on their 
individual skills. The lack of individual skill training isn’t as critical though, as it’s 
unlikely that there would be much positive transfer of training for a soldier using a 
keyboard and mouse to replicate their individual combat tasks. 
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Where this platform provides its best individual training is in reinforcing 
procedures. By putting the soldier in a situation that requires proper use of their unit’s 
Standard Operating Procedures (SOPs) the game is stressing the player’s cognitive 
abilities. For effective training on these SOPs though, it is critical that the squad’s 
leadership enforce proper procedures. Crossing danger areas (i.e. open terrain); 
providing suppressive fire; clearing an objective; reporting in status, enemy sightings, 
and other information to leaders, are all critical components of the individual soldier’s 
real world mission. Within the realm of this game, there is ample opportunity to execute 
these SOPs. Within the context of the elements of combat, we determined the following 
positive implications and areas of concern for COTS game use. 

1. Positive Potential Uses for COTS Games 

a. Performance of Unit SOPs. 

Having never worked together in a tactical environment, we began to 
develop SOPs as situations would present themselves. We would then use those SOPs 
throughout the exercise (i.e. crossing danger areas and assigning quadrants of fires on 
objectives) with positive success. The importance of this finding is, in conjunction with 
other live and constructive training events, this platform demonstrated the ability for units 
to train and rehearse SOPs well with a moderate level of fidelity in a VE. 

b. Brevity and Completeness of Tactical Reports. 

As the scenario unfolded, we became increasing reminded of the need for 
timely, concise and accurate reporting. We quickly develop a standard report format 
(which we enforced during game play) that was based on the standard Army SALUTE 
(Size, Activity, Location, Unit/Uniform, Time, and Equipment) format with brevity code 
words for ease of reporting. The result was rapid and concise reports that allowed us to 
constantly update our common operational picture. The COTS game provided a good 
tactical environment with adequate sensory (visual and auditory) cues to gather 
information and turn it into intelligence. 

c. Moderate Level of Support for Tactical Movement. 

Although the game provided some measure of spatial orientation, we did 
find that we had moderate cues from the VE to permit good training on tactical 
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movement lAW terrain, weather and the enemy. A larger study group will be required 
for further analysis. 

d. Direct Fire Plan (DFP) and Intelligence Preparation of the 
Battlefield (IPB). 

Although the game does not outright have the ability to export ground 
tactical information and an enemy template, the tools for developing the DFP and IPB 
can be gleaned from the game scenario ahead of time to develop a training exercise. This 
will require units taking screenshots of the scenario map ahead of time to develop 
independent graphics, IPB and a ground tactical plan. 

e. Command and Control. 

We found that the Delta Force provided a synthetic environment large 
enough to adequately maneuver small units lAW a ground tactical plan. This skill must 
be brought to the training system (as Delta Force alone will not train C ), but with this 
skill trained, the networked FPS platform could provide a good training tool for a user 
unit. 

2. Concerns for COTS Game Use as a Training Tool 

a. Communication. 

We found that tactical chat, the game’s replicated hand and arm signals 
and the lack of a voice over IP (VoIP) tool did not adequately support required 
communication methods. These shortcomings can be resolved by forcing units to use 
organic radio equipment for verbal communication or placing workstations close together 
to allow squad members to hear audio commands over the game’s sound effects. 

b. Audio. 

Performance of the Sound Localization and Hearing Sensitivity human 
abilities, as defined by the 0*NET Descriptors, was improved dramatically with the use 
of stereo headphones. Without these, the ability to localize sounds was problematic. 

c. Map Reading. 

Delta Force’s map function shows real-time locations of all player units. 
Although helpful for IFF (Identification of Friendly Forces), it lends itself to encourage 
users to rely more on the map’s IFF than keeping track of squad members through visual 
and auditory cues. The augmented map is a good example of a tool that could be a 
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scaffold, and ratcheted down as the players become familiar with tracking unit members 
in the virtual environment. 

d. Navigation Waypoints Display : 

The HUD display for navigation waypoints in Delta Force provides an 
adequate replication of the display of the AN/PSN-11 PLGR (Precision Lightweight 
(GPS) Receiver); however, only key leaders in an infantry unit carry a PLGR, not every 
Soldier. Again, this could be used as a scaffold for team members to be removed once a 
level of confidence is built in navigation of the VE. 

e. Weapon Performance. 

Delta Force’s weapons effects and performance do not accurately reflect real life 
performance. First and foremost, the weapons are uncharacteristically accurate at long 
distances. The weapon has an uncanny “first round on target” capability that does not 
reflect the real world conditions of Soldier fatigue, wind, obstacles, weapon malfunction 
and numerous other weapons performance limitations. Additionally, every round of 
ammunition fired (enemy and friendly) is a tracer. In reality, only key leaders carry 
magazines of tracers (for identification of key targets to engage for their team members 
and crew served weapons) and the M249 SAW (Squad Automatic Weapon) basic load 
consists of 3:1 mix (ball/tracer) of 5.56-linked ammunition. Although these 
shortcomings do not adversely affect the platform’s ability to train collective tasks, there 
is a real concern that users will become overconfident in their real weapons performance 
based on the level of accuracy of the game. 

f Graphics. There is no natural environmental visual static to draw 
attention away from a hidden or unmoving enemy. As a result, the only moving objects 
in the VE are enemy or friendly forces. For conducting defensive operations, this could 
be a potential problem where a defending force will have an unrealistic advantage in 
scanning without having to contend with blowing sand, animals, glare, or swaying of 
grass or trees. 


157 



THIS PAGE INTENTIONALLY LEET BLANK 


158 



INITIAL DISTRIBUTION LIST 


1. Defense Technical Information Center 
Ft. Belvoir, VA 

2. Dudley Knox Library 
Naval Postgraduate School 
Monterey, CA 

3. Major General Paul D. Eaton 
Deputy Chief of Staff, Operations 
Training and Doctrine Command 
Ft. Monroe, VA 

4. ETC Geoff Ellerson 
Commander, 2-11 Infantry (lOBC) 

Ft. Benning, GA 

5. ETC Favio Eopez 

Deupty Chief of Staff, G-3/5/7, ATTN: DAMO-SB 
Headquarters, Department of the Army 
Washington, DC 

6. Brent Houston 
NovaEogic, Inc. 

Calabasas, CA 


159 



